Machine learning enhances risk stratification and treatment failure prediction in diffuse large B‐cell lymphoma

Abstract

Diffuse large B‐cell lymphoma (DLBCL) is the most common lymphoma subtype worldwide. Existing prognostic models, including the National Comprehensive Cancer Network International Prognostic Index (NCCN IPI), rely on small predefined variable sets, discretized inputs, and do not incorporate longitudinal clinical histories or nonlinear relationships. To address these limitations, we present $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$, a machine learning model developed using clinical and laboratory data from 14,832 patients from the Danish Lymphoid Cancer Research (DALY‐CARE) resource (2005–2021). The primary objective was fixed‐time risk prediction of treatment failure within 2 years of first‐line therapy. Compared with the NCCN IPI at matched specificity, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ improved recall relatively by 22% (0.32–0.39), precision by 7% (0.56–0.60), and precision–recall AUC by 28% (0.46–0.59) in a blinded control population. Survival analyses were performed as a complementary evaluation and demonstrated reduced age‐related bias in risk stratification. $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ identified more than four times as many low‐risk patients compared to the NCCN IPI, largely by correctly classifying older patients with favorable outcomes. Prognostic performance was nearly identical when evaluated on DLBCL alone and on a cohort including multiple aggressive lymphoma subtypes, suggesting that shared prognostic signals can be leveraged through joint modeling. These findings show that addressing key limitations of existing prognostic models enables more accurate and individualized risk stratification in DLBCL. $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ provides a general framework for registry‐based prognostic modeling that can be adapted to other lymphoma subtypes and healthcare systems.

INTRODUCTION

Diffuse large B‐cell lymphoma (DLBCL) is the most common lymphoma worldwide. ¹ Although outcomes have improved over recent decades with the introduction of rituximab and CHOP‐like (e.g., CHOP, MAXICHOP, and CHOEP) chemotherapy regimens, ² , ³ 30%–50% of patients still experience treatment failure. ⁴ , ⁵ Early risk assessment of patients is important for treatment selection and clinical trial enrollment, since frontline therapy is often guided by disease stage and prognostic indices. ¹ , ⁶ , ⁷

Risk assessment in DLBCL remains dominated by the International Prognostic Index (IPI). ⁸ While several refinements have been proposed in the rituximab era, ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ the National Comprehensive Cancer Network International Prognostic Index (NCCN IPI) ⁹ is generally regarded as the most robust risk stratification tool. ¹¹ , ¹⁶ Nevertheless, both the IPI and the NCCN IPI are limited by relying on a small set of baseline diagnostic variables and on discretization of continuous variables, ⁸ , ⁹ despite evidence that inclusion of long‐term histories and continuous measures can improve discrimination. ¹⁴ In addition, survival‐based indices may overweight the contribution of age ¹⁷ and competing mortality in contemporary cohorts, potentially masking other clinically relevant disease‐related factors. ¹⁸

In contrast, modern machine learning (ML) approaches can alleviate many of these limitations by incorporating larger numbers of predictors, capturing nonlinear relationships, and learning from longitudinal patient histories now routinely stored in electronic health records (EHRs). ¹⁹ Prior studies have shown that such methods can outperform rule‐based prognostic scores in heterogeneous diseases such as DLBCL ²⁰ , ²¹ , ²² and support real‐time, individualized risk estimation. ²³

In this study, we use ML as a targeted strategy to overcome key limitations of existing prognostic scoring systems, such as IPI and NCCN IPI. Specifically, we evaluate whether ML‐based models trained on longitudinal, routinely collected health data can improve fixed‐time prediction of treatment failure among patients receiving CHOP‐like first‐line therapy. We further examine whether joint training across multiple aggressive lymphoma subtypes can maintain or enhance prognostic performance, reflecting the hypothesis that related diseases share clinically relevant prognostic signals. Similar cross‐disease modeling strategies have shown promise in oncology, including pan‐cancer mortality prediction. ²⁴ Together, this work aims to provide a more empirically grounded and flexible risk stratification framework that others can build upon as both data sources and computational methods continue to evolve. ⁷ , ¹⁴ , ¹⁶

METHODS

A consort diagram detailing our inclusion and exclusion criteria is shown in Figure 1. We initially identified 22,115 patients registered in the Danish National Lymphoma Registries (LYFO) ²⁵ who received a lymphoma diagnosis from 2005 to 2023. After applying our exclusion criteria (Supporting Information S1: A.1), our cohort consisted of 14,832 eligible patients who initiated first‐line treatment before 2022, ensuring at least two years of follow‐up. Data were split into a training set (85%) and a blinded test set (15%), stratified by death, relapse, and lymphoma subtype. A small number of patients (<1%) were registered with a preliminary identification number, which precluded linkage to national demographic registries. These patients were excluded from all analytic data sets (Supporting Information S1: A.1).

Figure 1.

Consort diagram of patients. We excluded patients with no first‐line treatment information, as we could not define a prediction point for these patients. We thereafter removed patients who received treatment after 2021 to ensure that we had a full 2‐year follow‐up for all patients included in the study. Patients registered with preliminary identification numbers (n = 134) were excluded from the analytic train and test sets due to missing demographic linkage. The subtypes of lymphomas included were diffuse large B‐cell lymphoma (DLBCL), follicular lymphoma (FL), classical Hodgkin lymphoma (cHL), marginal zone lymphoma (MZL), mantle cell lymphoma (MCL), Waldenstrom macroglobulinemia (WM), non‐Hodgkin lymphoma not otherwise specified (nHL), small lymphocytic lymphoma (SLL), and lymphocyte‐predominant Hodgkin's disease (HD‐LP). See Supporting Information S1: A.1 for further details.

For the primary evaluation, we assessed model performance in DLBCL patients receiving CHOP‐like first‐line therapy in the test set. Unless otherwise specified, performance metrics are reported for this population (Figure 1). Sensitivity analyses were performed for patients receiving non‐CHOP‐like treatments, patients receiving R‐CHOP‐like treatments, and all patients regardless of treatment (Supporting Information S1: B.6).

We defined a composite endpoint of treatment failure as either death, relapse, or initiation of second‐line treatment within 2 years of first‐line treatment. The time of treatment failure was the earliest occurrence of any of these events. Performance metrics for death and relapse as individual outcomes, as well as models trained specifically on each endpoint, are reported in Supporting Information S1: B.5. As a sensitivity analysis, we evaluated models using overall survival as the endpoint without retraining, with results presented in Supporting Information S1: C.10.

Data sources

For all patients, we gathered data from 8 different data sources using the Danish Lymphoid Cancer Research (DALY‐CARE) data resource ²⁶ : prescription medication, laboratory tests, surgical and other procedures, pathology codes, diagnosis codes, blood cultures, microbiology charts, and quality registers ²⁵ , ²⁷ (Figure 2).

Figure 2.

Schematic overview of our data‐driven feature generation process. (A) Using the quality registers, we identify relevant patients and define outcomes. We then extract data from the other seven different data sources. (B) These data sources are then processed to create different features. An example of laboratory tests is used to illustrate our approach. If a variable is recorded for more than 10% of patients in the training data within 3 years before first‐line treatment, we aggregate the data within all lookback (LB) windows using the relevant aggregation functions for each data source. This process results in 3881 candidate features. (C) Using lasso regression, we eliminate uninformative features. We reinclude the variables used by the NCCN IPI if they were removed by the LASSO regression procedure. This results in a final feature matrix with 87 features.

Feature creation and selection

We developed a data‐driven pipeline to systematically extract features from the longitudinal patient histories across all EHR modalities (Figure 2). We accounted for the irregular time series of EHR data by using the timeseriesflattener framework ²⁸ to aggregate predictors within different lookback windows before initiation of first‐line treatment. For all 7 EHR data modalities (excluding the quality registry), we included three different lookback windows: 3 months, 1 year, and 3 years (Supporting Information S1: A.2 and Supporting Information S1: Table 3). Subsequently, we used lasso regression for feature selection, resulting in a final set of 87 features used for model training and evaluation (Supporting Information S1: Table 4). All NCCN IPI variables were retained regardless of LASSO selection.

Model design overview

To investigate how feature complexity and training population affect clinical risk prediction, we developed three ML models with increasing scope:

$\mathbf{M}{\mathbf{L}}_{\mathbf{IPI}}$ used only the continuous versions of the five NCCN IPI variables to isolate the effect of modeling technique. $\mathbf{M}{\mathbf{L}}_{\mathbf{DLBCL}}$ used 87 features from patient records but was trained solely on DLBCL cases to assess whether disease‐specific training improves performance. $\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$, the primary model, used the same feature set of 87 features but was trained across all aggressive lymphoma subtypes to leverage shared clinical patterns and potential gains from pooled learning (Supporting Information S1: Table 1).

To further explore model specialization, we trained subtype‐specific models for other lymphomas (e.g., FL, MCL, and cHL) using the same 87 features. These models permitted direct comparison of subtype‐specific training with broader pooled training. Subtype‐specific models are described in detail in Supporting Information S1: A.5.

Decision thresholds for all models are denoted using superscripts (e.g., $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.5}$ for a 0.5 threshold). Adjusting the decision thresholds allows for more targeted comparisons with, for instance, matching the recall or precision of another model (e.g., NCCN IPI). All models were trained using the XGBoost algorithm, ²⁹ a gradient boosting framework well‐suited for structured clinical data. ³⁰ Model hyperparameters are provided in Supporting Information S1: A.4.

Baseline comparisons

The NCCN IPI was used as the main clinical comparator to the ML models. Our primary objective was fixed‐time risk prediction, where the goal was to estimate the probability of treatment failure within two years of first‐line therapy. As the NCCN IPI is applied in practice as a categorical risk score rather than a probability model, we evaluated it both as a binary classifier and as a risk stratification tool.

For binary classification, patients with an NCCN IPI score of 6 or higher were considered high risk (treatment failure), while all others were classified as low risk (treatment success). For threshold‐independent metrics such as area under the precision–recall curve (PR‐AUC), each integer score was treated as a decision threshold. Uncertainty estimates for performance metrics were obtained using nonparametric bootstrapping with 1000 resamples of the test set to derive 95% confidence intervals. Statistical significance of pairwise differences in performance was assessed using paired bootstrap resampling ³¹ , ³² (Supporting Information S1: A.9).

To enable comparison of risk stratification, we grouped patients into four risk categories based on the predicted probability of treatment failure estimated by $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$. These categories were designed to align with the four NCCN IPI risk groups. Cutoffs were selected according to two criteria: (i) the “Low”‐risk group should have a comparable event rate to the NCCN IPI's low‐risk group and (ii) each group should be large enough to be clinically meaningful. While these cutoffs are somewhat arbitrary and can be adjusted for specific use cases, we used the following thresholds: <0.1 for “Low,” 0.1–0.3 for “Low‐Intermediate,” 0.3–0.65 for “Intermediate‐High,” and ≥0.65 for “High.”

Survival analysis

The NCCN IPI was originally developed for risk stratification of time‐to‐event outcomes, such as overall and progression‐free survival at five years, rather than for fixed‐time binary classification of treatment failure. ⁹ To allow comparison within that framework and to assess how $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$risk groups behave over time, we examined Kaplan–Meier Estimates (KMEs) ³³ of treatment‐free survival across the corresponding risk categories. KMEs are reported at two years after first‐line treatment in the main text, and at five and ten years in Supporting Information S1: C.3 and C.6. To evaluate calibration and discrimination in a time‐to‐event setting, we fit Cox proportional hazards models using risk group assignment as the sole predictor and computed concordance indices and Integrated Brier scores based on these models.

Explainability

To explain what variables contributed to the predictions made by the $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ model, we calculated Shapley Additive Explanations (SHAPs) ³⁴ across all features for patients in the test set. We provide SHAP analyses for the $\mathrm{M}{\mathrm{L}}_{\mathrm{DLBCL}}$ and $\mathrm{M}{\mathrm{L}}_{\mathrm{IPI}}$ in Supporting Information S1: C.5.2.

Data sharing and reproducibility

The data used for this project are a subset of the Danish Lymphoid Cancer Research (DALY‐CARE) data resource. We refer to the paper ²⁶ introducing the data resource for details concerning the data and how to access them. All scripts and feature engineering pipelines for this project are publicly available on GitHub to enable replication and testing in external cohorts. To support easier clinical integration and external validation, we derived a more minimal model using recursive feature elimination (RFE), retaining only 30 features (Supporting Information S1: Table 4) while preserving near‐equivalent performance to the full model (Supporting Information S1: C.8). This reduced model is well‐suited for settings with more limited data availability.

RESULTS

A detailed overview of clinical characteristics for the included patients is shown in Table 1, demonstrating similar characteristics between the training and the unseen test set. At the same time, age and risk factors differ between CHOP‐like‐treated DLBCL patients and DLBCL patients receiving other treatments.

Table 1.

Summary of baseline characteristics for the 14,698 patients included in the analytical training and test set, stratified by diagnosis and first‐line treatment regimen.

		DLBCL + CHOP‐like		DLBCL ‐ CHOP‐like		Other lymphomas
		N = 5848		N = 1209		N = 7641
		Train	Test	Train	Test	Train	Test
		N = 4951	N = 897	N = 1046	N = 163	N = 6490	N = 1151
Age	≤40	265 (5.35)	52 (5.80)	31 (2.96)	5 (3.07)	918 (14.14)	155 (13.47)
	41–60	1128 (22.78)	213 (23.75)	178 (17.02)	21 (12.88)	1578 (24.31)	268 (23.28)
	61–75	2259 (45.63)	401 (44.70)	393 (37.57)	66 (40.49)	2562 (39.48)	461 (40.05)
	>75	1299 (26.24)	231 (25.75)	444 (42.45)	71 (43.56)	1432 (22.06)	267 (23.20)
Sex	Males	2804 (56.64)	519 (57.86)	588 (56.21)	83 (50.92)	3638 (55.06)	630 (54.74)
AA stage	I	941 (19.01)	194 (21.63)	406 (38.81)	63 (38.65)	1221 (18.18)	220 (19.11)
	II	859 (17.35)	148 (16.50)	96 (9.18)	10 (6.13)	1046 (16.12)	185 (16.07)
	III	964 (19.47)	177 (19.73)	100 (9.56)	18 (11.04)	994 (15.32)	161 (13.99)
	IV	2143 (43.28)	369 (41.14)	408 (39.01)	67 (41.10)	3155 (48.61)	581 (50.48)
	NA	44 (0.89)	9 (1.00)	36 (3.44)	5 (3.07)	74 (1.14)	4 (0.35)
ECOG PS	≥2	808 (16.32)	146 (16.28)	391 (37.39)	65 (39.88)	565 (8.71)	104 (9.03)
	NA	21 (0.42)	1 (0.11)	3 (0.29)	0 (0.00)	22 (0.34)	3 (0.26)
B Symptoms	Present	2015 (40.70)	378 (42.14)	336 (32.12)	69 (42.33)	2293 (35.33)	398 (34.59)
	NA	119 (2.40)	18 (2.01)	74 (7.07)	9 (5.52)	109 (1.68)	25 (2.17)
LDH	≤ULN	2203 (44.50)	410 (45.71)	504 (48.18)	83 (50.92)	4346 (67.86)	781 (67.85)
	1–3 × ULN	2174 (43.91)	395 (44.04)	433 (41.40)	53 (32.52)	1874 (28.88)	327 (28.41)
	>3 × ULN	452 (9.13)	73 (8.14)	75 (7.17)	24 (14.72)	103 (1.59)	14 (1.22)
	NA	122 (2.46)	19 (2.12)	34 (3.25)	3 (1.84)	167 (2.57)	29 (2.52)
ALC	≤0.84 × 109/L	1124 (22.70)	193 (21.52)	319 (30.50)	48 (29.45)	868 (13.37)	164 (14.25)
	NA	202 (4.08)	30 (3.34)	46 (4.40)	10 (6.14)	248 (3.82)	35 (3.04)
Albumin	≤35 g/L	1482 (29.93)	251 (27.98)	407 (38.91)	68 (41.72)	1480 (22.80)	261 (22.68)
	NA	825 (16.66)	166 (18.51)	188 (17.97)	28 (17.18)	1053 (16.22)	186 (16.16)
HGB	<10 g/dL	551 (11.13)	92 (10.26)	98 (9.37)	26 (15.95)	764 (11.77)	146 (12.68)
	NA	35 (0.71)	5 (0.56)	14 (1.34)	2 (1.23)	49 (0.76)	10 (0.87)
Bulky disease	≥10 cm	914 (18.46)	171 (19.06)	122 (11.66)	17 (10.43)	669 (10.31)	111 (9.64)
	NA	1191 (24.04)	219 (24.41)	323 (30.88)	52 (31.9)	1857 (28.61)	350 (30.41)
IPI	0	366 (7.39)	75 (8.36)	47 (4.49)	5 (3.07)	871 (13.42)	145 (12.60)
	1	1029 (20.78)	192 (21.40)	216 (20.65)	34 (20.86)	1609 (24.79)	286 (24.85)
	2	1189 (24.02)	215 (23.97)	246 (23.52)	41 (25.15)	2008 (30.94)	373 (32.41)
	3	1194 (24.12)	218 (24.30)	253 (24.19)	44 (26.99)	1302 (20.06)	238 (20.68)
	4	757 (15.29)	130 (14.49)	150 (14.34)	20 (12.27)	384 (5.92)	61 (5.30)
	5	228 (4.61)	39 (4.35)	65 (6.21)	12 (7.36)	55 (0.85)	10 (0.87)
	NA	188 (3.80)	28 (3.12)	69 (6.60)	7 (4.29)	261 (4.02)	38 (3.30)
NCCN IPI	0	52 (1.05)	12 (1.34)	0 (0.00)	1 (0.61)	362 (5.58)	53 (4.60)
	1	258 (5.21)	52 (5.80)	15 (1.43)	1 (0.61)	596 (9.18)	106 (9.21)
	2	583 (11.78)	101 (11.26)	59 (5.64)	8 (4.91)	999 (15.39)	183 (15.90)
	3	953 (19.25)	188 (20.96)	180 (17.21)	20 (12.27)	1419 (21.86)	253 (21.98)
	4	1126 (22.74)	206 (22.97)	260 (24.86)	50 (30.67)	1490 (22.96)	282 (24.50)
	5	984 (19.87)	171 (19.06)	240 (22.94)	38 (23.31)	995 (15.33)	179 (15.55)
	6	585 (11.82)	105 (11.71)	144 (13.77)	23 (14.11)	307 (4.73)	48 (4.17)
	7	201 (4.06)	28 (3.12)	66 (6.31)	9 (5.52)	58 (0.89)	11 (0.96)
	8	28 (0.57)	7 (0.78)	13 (1.24)	6 (3.68)	7 (0.11)	0 (0.00)
	NA	181 (3.66)	27 (3.01)	69 (6.60)	7 (4.29)	257 (3.96)	36 (3.13)

Note: Patients are divided into training and held‐out test sets across three cohorts: (1) DLBCL patients treated with CHOP‐like regimens, (2) DLBCL patients treated with other regimens, and (3) patients with other lymphoma subtypes. Percentages are column‐wise within each subgroup. Missing values are reported as NA.

Abbreviations: AA stage, Ann Arbor Stage; ALC, Absolute Lymphocyte Count; ECOG PS, ECOG performance status; HGB, hemoglobin; IPI, International Prognostic Index; LDH, lactate dehydrogenase; NCCN IPI, National Comprehensive Cancer Network IPI; ULN, Upper Limit of Normal.

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$ outperforms NCCN IPI across all considered metrics

We compared the performance of $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ with the NCCN IPI in CHOP‐like‐treated DLBCL patients in the test set. $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.5}$ outperformed the NCCN IPI across all considered threshold‐independent and classification metrics (PR‐AUC: 0.59 vs. 0.46; ROC‐AUC: 0.80 vs. 0.72; MCC: 0.34 vs. 0.27) (Table 2). Additional comparator models, including a logistic regression baseline and the transformer‐based TabPFN ³⁵ classifier, outperformed the NCCN IPI but showed poorer performance than $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ across all performance metrics (Supporting Information S1: Material B.7). As a secondary evaluation, we assessed prediction of relapse and death in isolation. Performance for relapse‐only prediction was consistently lower than for the composite endpoint across all models, with only marginal gains from outcome‐specific training (Supporting Information S1: B.5, C.1).

Table 2.

Performance metrics on the test set.

Metric	NCCN IPI	$\mathrm{M}{\mathrm{L}}_{\text{IPI}}^{0.5}$	$\mathrm{M}{\mathrm{L}}_{\text{DLBCL}}^{0.5}$	$\mathrm{M}{\mathrm{L}}_{\text{All}}^{0.5}$
PR‐AUC	0.46 (0.42–0.51)	0.51 (0.46–0.57) ★	0.58 (0.53–0.64) ★	0.59 (0.52–0.63) ★
ROC‐AUC	0.72 (0.69–0.76)	0.74 (0.71–0.78)	0.79 (0.76–0.82) ★	0.80 (0.77–0.82) ★
MCC	0.27 (0.20–0.34)	0.28 (0.21–0.34)	0.37 (0.30–0.44) ★	0.33 (0.26–0.40)
Precision	0.56 (0.49–0.64)	0.57 (0.50–0.64)	0.63 (0.57–0.69) ★	0.60 (0.54–0.67)
Recall	0.32 (0.26–0.37)	0.33 (0.27–0.39)	0.43 (0.37–0.49) ★	0.39 (0.33–0.44) ★
Specificity	0.90 (0.88–0.92)	0.90 (0.88–0.92)	0.90 (0.88–0.92)	0.90 (0.87–0.92)

Note: Superscripts for ML models refer to the decision threshold used for classification. The subscript refers to different models. Bold numbers indicate the highest scores across all models for the specific performance metric. A star (★) denotes a statistically significant improvement compared with the NCCN IPI, based on paired bootstrap resampling (2000 replicates). Differences were considered significant when the 95% bootstrap confidence interval for ∆ excluded 0. Note that metrics for NCCN IPI are only calculated for patients who did not have missing values in the variables used to calculate the indices.

Abbreviation: MAC, the Matthews correlation coefficient.

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$ improves precision–recall trade‐offs compared with NCCN IPI

At a fixed specificity, corresponding to the NCCN IPI high‐risk threshold, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.5}$ improved recall from 0.32 to 0.39 (22%) and precision from 0.56 to 0.60 (7%). The precision–recall curve for $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ was consistently above that of the NCCN IPI across all thresholds (Figure 3). Varying the classification threshold allowed for different trade‐offs between recall and precision: $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.4}$ increased recall from 0.32 to 0.51 while maintaining NCCN IPI's precision; $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.59}$ increased precision from 0.56 to 0.67 at the recall level of NCCN IPI.

Figure 3.

Precision–recall curve of the NCCN IPI and ML_All . The legend shows the average precision (AP) as measured as the area under the curve. The bottom left arrow shows the “High”‐risk group from NCCN IPI and captions report the precision and recall of the model. The other two arrows point to $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ thresholds that match either the recall (upper left) or the precision (lower right) of the NCCN IPI “High”‐risk group.

To quantify the clinical implications of improved precision, we calculated the number needed to treat (NNT). At matched recall, the high‐risk group identified by $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.59}$ had a 67.5% event rate versus 56.4% for the NCCN IPI. This corresponds to an effective NNT of 9.0; for every 9 patients flagged as high risk by $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$, one additional treatment failure is correctly identified compared to the NCCN IPI.

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$ improves performance by addressing limitations of traditional models

To disentangle the sources of this improvement, we conducted an ablation‐style analysis. A minimal model, $\mathrm{M}{\mathrm{L}}_{\mathrm{IPI}}^{0.5}$, using the same variables as NCCN IPI but retaining continuous values, improved PR‐AUC from the NCCN IPI's 0.46 to 0.51. Adding broader features and longitudinal histories in $\mathrm{M}{\mathrm{L}}_{\mathrm{DLBCL}}^{0.5}$ increased PR‐AUC to 0.58. Joint training in $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}^{0.5}$ further increased PR‐AUC to 0.59.

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$provides more granular risk stratification than NCCN IPI

Using the derived risk groups as predictors in time‐to‐event analyses, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ showed higher concordance and better calibration than the NCCN IPI. The C‐index was 0.71 (95% CI: 0.68–0.73) for $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ compared with 0.65 (95% CI: 0.62–0.68) for the NCCN IPI. Similarly, the Integrated Brier score was lower for $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ (0.165 [95% CI: 0.153–0.177]) than for the NCCN IPI (0.182 [95% CI: 0.170–0.195]).

Based on these risk groups, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$identified more than four times as many low‐risk patients compared to NCCN IPI while maintaining comparable treatment failure rates (Figures 4 and 5). Kaplan–Meier estimates showed clearer separation between $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ risk groups, particularly between the “Low” and “Low‐Intermediate” categories (Figure 5).

Figure 4.

Scatterplot of predictions made by the NCCN IPI and ML_All . The x‐axis shows the estimated probability from the ML_All model and the y‐axis shows the risk groups identified by the NCCN IPI. Points show individual patients and are colored by whether they had treatment failure within 2 years (red) or not (blue). The red background shading shows the different risk groups created based on the estimated probability of the ML_All model.

Figure 5.

Kaplan–Meier curves and estimations for different risk groups. Only patients in the test set with a DLBCL diagnosis who received CHOP‐like treatment were included in this analysis. The first panel shows Kaplan–Meier curves over time. The dashed vertical line highlights the time point 2 years after treatment. Shaded areas show the 95% CI around probability estimates. Crosses on lines show censored patients. Median follow‐up times for the NCCN IPI risk groups were 9.78 years for the “Low”‐risk group, 9.46 years for the “Low‐Intermediate”‐risk group, 8.58 years for the “Intermediate‐High”‐risk group, and 7.70 for the “High”‐risk group. Median follow‐up times for the ML_All risk groups were 8.79 years for the “Low”‐risk group, 10.28 years for the “Low‐Intermediate”‐risk group, 9.60 years for the “Intermediate‐High”‐risk group, and 6.67 years for the “High”‐risk group. The second panel shows a forest plot of the Kaplan–Meier estimates (with estimates converted into percentages) two years after first‐line treatment accompanied by 95% confidence intervals in parentheses.

To examine agreement at the patient level, we performed a concordance–discordance analysis (Supporting Information S1: C.9.1). Patients classified as higher risk by $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ but lower risk by the NCCN IPI had poorer 2‐year treatment‐free survival (54% [95% CI: 37%–78%]), whereas those classified as lower risk by $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ but higher risk by the NCCN IPI had more favorable outcomes (77% [95% CI: 72%–82%]).

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$reduces age‐related bias in risk stratification

The NCCN IPI risk score showed a stronger correlation with patient age ($\rho$ = 0.54) than the continuous predicted probabilities of $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ ($\rho$ = 0.37). Among patients classified as high risk, the 2‐year treatment‐free survival rate was similar across older age strata for $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ (60–75 years: 30% [95% CI: 19%–45%] vs. >75 years: 29% [95% CI: 19%–45%]), whereas the NCCN IPI showed greater variation (57% [95% CI: 45%–71%] vs. 32% [95% CI: 23%–45%)] (Figure 6). $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ identified 160 low‐risk patients over 60 years of age, whereas the NCCN IPI identified only one. Additional age‐stratified analyses are presented in Supporting Information S1: C.3. Kaplan–Meier estimates of treatment‐free survival at 2, 5, and 10 years across risk groups and cohorts are provided in Supporting Information S1: C.6.

Figure 6.

Heatmaps of Kaplan–Meier estimates and counts grouped by age and risk groups. Only DLBCL patients who received CHOP‐like treatment in the test set were included. The x‐axis shows different risk groups and the y‐axis shows different age groups. Left panels show results from $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ and the right panel shows the results from NCCN IPI. The numbers and colors in the upper panels show Kaplan–Meier estimates for treatment‐free survival (TFS) 2 years after first‐line treatment with 95% CIs in parentheses. The numbers and colors in the lower panels show the number of patients in each group. Missing squares indicate that no patients were in both the age and risk group.

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$ shows stable performance under joint modeling

In the cohort including all lymphoma patients, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ achieved a PR‐AUC of 0.61 (95% CI: 0.57–0.64) and a ROC‐AUC of 0.79 (95% CI: 0.77–0.81), similar to its performance in CHOP‐like‐treated DLBCL patients (PR‐AUC: 0.59; 95% CI: 0.52–0.63; ROC‐AUC: 0.80; 95% CI: 0.77–0.82) (Supporting Information S1: Table 10 and Table 2). Kaplan–Meier estimates also showed consistent stratification when performed on all lymphoma patients (Supporting Information: C.6). Training jointly across lymphoma subtypes generally improved prognostic performance compared to subtype‐specific models (Figure 7 and Supporting Information S1: Table 14).

Figure 7.

Comparison between jointly trained and subtype‐specific models. Each point compares the AU‐PRC of a subtype‐specific model (x‐axis) with that of the pooled model, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ (y‐axis), for a given lymphoma subtype. Point size reflects the number of patients per subtype. Points above the diagonal indicate better performance of the $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ model. Arrows indicate performance directionality. $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ achieves comparable or superior performance in most subtypes despite not being tailored to any one group, supporting its utility as a generalizable risk stratification tool. Figure adapted from Bjerregaard‐Michelsen et al. (2025).

In WM, the subtype‐specific model showed higher performance than $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$. Across all other subtypes, differences in PR‐AUC and ROC‐AUC were never more than 0.02 lower for $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ than the best subtype‐specific models and were often substantially higher (Figure 7 and Supporting Information S1: B.6). Full subtype‐specific performance results, as well as analyses of all DLBCL patients and DLBCL patients not receiving CHOP‐like regimens, are reported in Supporting Information S1: B.6.

$\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$ captures prognostic signals across clinical domains and time windows

Feature importance was assessed using absolute SHAP values. Core NCCN IPI variables such as age, LDH, and ECOG performance status were among the most influential features. Additional disease‐specific factors such as tumor diameter, hemoglobin levels, and platelet counts also ranked highly. Markers of healthcare interactions, including referrals for radiological procedures and outpatient visits, were also associated with model predictions (Figure 8). Feature importance was stable across training and test sets (Supporting Information S1: C.5.1).

Figure 8.

SHAP plot for the 20 most important features for the $\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$ model as measured by absolute mean SHAP values. Each point represents a single patient; the x‐axis shows the SHAP value (the feature's contribution to increasing or decreasing the predicted probability of treatment failure) and the y‐axis lists the feature names. Point color reflects the individual feature value (red = higher value, blue = lower value; and gray = missing). Feature names include the aggregation function (e.g., count) if not a static baseline feature, and the lookback window used for feature generation is indicated in parentheses. Sex was encoded as 0 and 1, with males being 1.

DISCUSSION

Our results show that addressing key limitations of existing prognostic models leads to meaningful improvements in predicting treatment‐free survival in DLBCL. Together, these findings support a move toward data‐driven prognostic models that better reflect real‐world patient trajectories.

In the sections that follow, we consider the broader implications of our framework alongside the methodological, practical, and clinical factors relevant to its interpretation and future application.

Addressing key limitations of traditional prognostic models

Across all considered evaluation metrics, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ outperformed the NCCN IPI (Table 2 and Supporting Information S1: C.9.1). The ablation‐style analyses indicate that these gains arise from progressively addressing key limitations of existing prognostic models. Models that retained raw clinical information within a flexible modeling framework showed improved performance, with further gains observed when longitudinal data from multiple modalities were incorporated (Table 2 and Supporting Information S1: Table 1). This is notable, as most clinical prognostic models rely exclusively on baseline measurements. ⁸ , ³⁶ , ³⁷ While the absolute gains in population‐level metrics such as ROC‐AUC and PR‐AUC may appear modest, the ability to generate more nuanced and individualized risk estimates is clinically important.

We view this work as one step in an iterative development process. By showing that prognostic performance improves when specific modeling limitations are addressed, our results outline a clear path for continued refinement. Further gains are likely achievable through targeted changes in data representation and model design to overcome additional limitations.

Limitations of aggregated longitudinal representations

Although our aggregated approach improved prognostic accuracy, it discarded much of the temporal information. As a result, we likely underestimate the full value of long‐term histories. In our benchmarking (Supporting Information S1: B.7), gradient‐boosted trees (XGBoost) outperformed logistic regression and the transformer‐based TabPFN classifier. ³⁵ However, TabPFN, like our current model, operates on aggregated features rather than raw temporal sequences, and its performance should therefore not be taken as evidence against deep learning approaches in this setting. The greatest potential of deep learning methods such as recurrent neural networks ³⁸ or attention‐based models ³⁹ lies in capturing temporal dependencies across a full disease trajectory of a patient. Our findings therefore support explicit temporal learning, incorporating the whole patient trajectory as a logical next step. ⁴⁰

Improved risk stratification and age‐adjusted prognosis

By incorporating additional disease‐related variables, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ generated prognostic estimates that are less driven by age than the NCCN IPI. Despite identifying substantially more older patients as low risk, the model maintained strong discrimination between risk groups, including at five years after first‐line treatment (Supporting Information S1: C.3, C.6). Reduced reliance on age enabled identification of older patients with favorable biological and clinical profiles who may otherwise be classified as high risk. This may have implications for clinical trial eligibility and for the use of intensive treatment strategies in selected elderly patients who might otherwise be undertreated based on age alone. Future work should further distinguish treatment failure driven by toxicity from disease progression.

Clinical utility and use cases of $\mathbf{M}{\mathbf{L}}_{\mathbf{All}}$

Beyond informing future methodological work, this framework also addresses an emerging clinical need. As phase 3 trials of frontline bispecific antibodies and CAR‐T therapies begin to report results, there is growing demand for prognostic tools that can support patient selection and risk stratification. More accurate individualized risk estimates may help identify patients most likely to benefit from intensified strategies, while reducing unnecessary toxicity and cost for lower risk individuals. Pending external validation, a model like $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ could ultimately support trial enrichment and contribute to more individualized treatment strategies as new therapies are integrated into practice. We do note, however, that effective clinical implementation depends less on universal generalizability than on context‐aware adaptation and validation within each healthcare setting. ⁴¹

Expanding data modalities: Toward biological integration

A key limitation of existing prognostic models is their restricted feature scope. Incorporation of additional modalities, such as molecular or genomic data, represents a natural extension that may further refine risk stratification. For instance, molecular subgroupings such as COO, ⁴² LymphGen, ⁴³ and DLBCLass ⁴⁴ are known to associate with prognosis, but were not available for the full cohort in this study. Integration of such data could help link ML‐derived risk estimates more directly with tumor biology. That said, prior work in CLL has shown that registry‐based models can capture most of the prognostic signal without genomics, ⁴⁵ suggesting that molecular data may offer incremental, but valuable improvements.

Generalization across subtypes and learning from related diseases

Training across multiple lymphoma subtypes generally maintained or improved prognostic performance compared with subtype‐specific models (Figure 7 and Supporting Information S1: B.6). This suggests that subtypes share clinically relevant features and supports the idea that learning across related diseases via techniques like transfer learning can enhance performance, particularly for rarer subtypes with limited data. ²⁴ , ²⁶ , ⁴⁶ In this study, cross‐subtype learning was implemented in a simple form by pooling data across subtypes during model training. More advanced approaches such as meta‐learning ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ or disease‐similarity‐aware sampling ⁵¹ could further improve performance, particularly for underrepresented cohorts ⁵² , ⁵³ and in the presence of domain shifts. ²⁶ , ⁴⁷ , ⁵⁴ Accordingly, these results likely represent a conservative baseline: they demonstrate that cross‐subtype learning is viable and promising, ²⁴ and improvements can likely be gained by implementing targeted transfer‐learning strategies.

Model interpretability and handling of missing data

Using SHAP values to assess feature importance, we found features beyond the NCCN IPI variables that were highly informative (Figure 8). Many of these have been proposed as potential prognostic biomarkers for DLBCL disease severity, including platelet count, ⁵⁵ hemoglobin levels, ⁵⁶ , ⁵⁷ , ⁵⁸ and tumor diameter. ⁵⁹ , ⁶⁰ This confirms that improved performance partly reflects incorporation of a broader range of disease‐related features and clinical context, enabling more individualized risk estimation.

Highly ranked predictors also included indicators of clinical assessment (e.g., radiological procedures), comorbidity burden, and healthcare utilization. These likely function as proxies for disease complexity or diagnostic intensity rather than direct causal determinants of outcome. For example, frequent imaging may reflect closer surveillance in patients with more severe or uncertain disease rather than conferring benefit itself.

Importantly, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ does not require complete data for any single variable. Tree‐based algorithms can natively accommodate missing values, allowing predictions even when many features are partially unavailable. ²⁹ This contrasts with conventional scores such as the NCCN IPI, which depend on complete data for each component. Although feature importance may generate hypotheses for further study, the model is intended for fixed‐time risk prediction and probabilistic risk stratification, rather than for causal inference or treatment selection.

Generalizability and implementation across health care systems

While $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ demonstrates strong prognostic performance within the Danish registry setting, it should primarily be viewed as a proof of concept for how routinely collected health data can support individualized prognostication. Some predictors may reflect organization‐ or documentation‐specific factors. Applying this framework in other health systems would therefore require adaptation, including harmonizing variable definitions, adjusting preprocessing pipelines, and retraining or fine‐tuning the model on local data to account for regional and institutional differences. ²³ , ⁴¹

As others have previously noted, attempts to build universally generalizable models often result in systems that perform only moderately well across many settings, rather than optimally within a single one. ⁴¹ Accordingly, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ is best understood as a transferable framework for registry‐based prognostic modeling that can be extended and adapted to other health systems where comparable structured data exist. ⁴¹ To support this, all modeling code and preprocessing pipelines are openly shared to enable replication, retraining, and integration into local environments.

Although the present work is not intended as a deployable clinical tool, it delineates a clear path toward that goal. While the full set of structured variables used here may not yet be uniformly available across healthcare systems, this study illustrates the potential performance gains achievable when such collected data are leveraged comprehensively.

Similar registry‐based machine‐learning approaches have already been integrated into EHR systems in other hematologic settings. ¹⁹ , ²³ Together with the fact that the present framework operates entirely on routinely collected structured data and tolerates incomplete records through native handling of missingness, this suggests that implementation is feasible. ¹⁹ , ²³ Future integration into EHR systems would require prospective validation, but the present work demonstrates a technically feasible and transferable framework for registry‐based prognostic modeling.

CONCLUSION

This study demonstrates that addressing key limitations of traditional prognostic models enables more accurate and individualized prediction of treatment failure in DLBCL. By integrating longitudinal, structured EHR data within a flexible modeling framework, $\mathrm{M}{\mathrm{L}}_{\mathrm{All}}$ outperformed the NCCN IPI in predicting treatment‐free survival among patients receiving CHOP‐like therapy. Continuous risk estimates enabled clinically adaptable trade‐offs between precision and recall, supporting context‐specific risk stratification. Moreover, comparable performance in DLBCL‐only and broader lymphoma cohorts indicates that shared prognostic signals can be leveraged through joint modeling. Pending prospective validation, this framework may support clinical trial enrichment, treatment stratification, and earlier identification of patients at increased risk of treatment failure.

AUTHOR CONTRIBUTIONS

Mikkel Werling: Conceptualization; funding acquisition; methodology; visualization; writing—original draft; writing—review and editing; formal analysis; software; data curation; project administration; investigation; validation. Alexander D. Fuglkjær: Conceptualization; writing—review and editing; methodology; validation. Peter Brown: Conceptualization; writing—review and editing; data curation. Carsten U. Niemann: Funding acquisition; conceptualization; writing—review and editing; supervision; data curation; resources; project administration; validation. Rudi Agius: Funding acquisition; conceptualization; supervision; project administration; writing—review and editing; methodology; validation.

CONFLICT OF INTEREST STATEMENT

Carsten Utoft Niemann received research funding and/or consultancy fees from AstraZeneca, Janssen, AbbVie, BeiGene, Genmab, CSL Behring, Octapharma, Takeda, Eli Lily, MSD, and Novo Nordisk Foundation. Peter Brown received research funding and/or consultancy fees from Roche, AbbVie, and BMS. All other authors declare no competing interests to disclose.

ETHICS STATEMENT

The data are available as a collaborator on the DALY‐CARE project. For guidance on how to become a collaborator, please contact one of the corresponding authors. We refer to the paper introducing the data resource for details concerning the data and how to access it. All scripts and feature engineering pipelines are publicly available on GitHub to enable replication and testing in external cohorts.

FUNDING

The project was funded partially by the Alfred Benzon foundation and by Sygeforsikringen “Danmark” (grant number 2022‐0086). This work was based on data analyzed at the national infrastructure for personal medicine hosted at the Danish National Genome Center, which is supported by the Novo Nordisk Foundation (grant agreement NNF18SA0035348 and grant agreement NNF19SA0035486). This work was supported by the Danish Data Science Academy, which is funded by the Novo Nordisk Foundation (NNF21SA0069429) and VILLUM FONDEN (40516). The PERSIMUNE project contributed data and received funding from the Danish National Research Foundation (#126).

Supporting information

Supplemental Information.

DATA AVAILABILITY STATEMENT

The data used for this project are a subset of the Danish Lymphoid Cancer Research (DALY‐CARE) data resource. The data are available as a collaborator on the DALY‐CARE project. For guidance on how to become a collaborator, please contact one of the corresponding authors. The DALY‐CARE protocol has been approved by the Danish Health Data Authority and Danish National Ethics Committee (approvals P‐2020‐561 and 1804410, respectively).