Spider-MS: an individualized polyhedral prediction of multiple sclerosis prognosis

Abstract

Multiple sclerosis (MS) is a potentially disabling disease that shows marked variability in its severity and underlying mechanisms. Early prediction of patients at risk of specific unfavourable outcomes may be key to treatment stratification and management. Here, we propose a prognostic framework, called the Spider-MS model, to predict a wide range of relevant outcomes at the individual level, at symptom onset.

We included patients from the Barcelona first-attack cohort, i.e. with a first demyelinating attack suggestive of multiple sclerosis, younger than 50 years old at symptom onset and seen in the clinic within 3 months of the first attack, to build the Spider-MS model (original cohort). Patients with a first demyelinating attack from the Royal Melbourne Hospital were used to validate the model externally (validation cohort). All patients were prospectively assessed clinically, with the Expanded Disability Status Scale (EDSS) and relapse tracking, and through brain and, in some cases, spinal cord MRI. Spider-MS was built as a set of eight accelerated failure models with Weibull distribution, one for each outcome, including McDonald 2017 diagnosis, second attack, yearly rate of new T2 lesions > 2, relapse-associated worsening (RAW) at the first attack and at subsequent attacks, confirmed and sustained disability worsening, progression independent of relapse activity (PIRA) and EDSS 3.0. Model predictors included age, sex, first attack topography, brain and spinal cord lesions, CSF oligoclonal bands and percentage of time on high-/moderate-efficacy treatment before the outcome.

We included 1180 patients from Barcelona (mean age 32.37 years, 810 females) and 108 from Melbourne (32.41 years, 78 females). Median follow-up times were 10.80 and 11.10 years, respectively. In the original cohort, 797/1180 (67.5%) fulfilled the McDonald criteria, 121/1180 (10.3%) developed RAW at subsequent relapses and 290/1180 (24.6%) developed PIRA over the follow-up period. The prediction models reached moderate-high accuracy levels (Harrell’s C values: 0.653–0.823) and showed that older age, cord involvement at first attack, greater number of brain and cord lesions, presence of CSF oligoclonal bands and lower percentage of time on treatment predicted a greater risk of unfavourable outcomes, with varying effects depending on the outcome. When the Spider-MS model was applied to the external cohort, for all outcomes except for RAW at first or subsequent attacks, the prediction models reached moderate-high accuracy values (Harrell’s C = 0.623–0.766). In general, patients with the highest predicted risks experienced acute inflammatory activity or disability earlier than lower-risk patients.

In conclusion, our Spider-MS model can be considered a promising individual predictive tool with the potential to inform clinical practice.

Tur et al. introduce Spider-MS, a prognostic model designed to predict how multiple sclerosis will progress in an individual after a first demyelinating attack. Built using large amounts of clinical and MRI data, Spider-MS has the potential to improve risk stratification and support more personalised treatment decisions.

Introduction

Multiple sclerosis (MS) is a potentially disabling disease with a high variability across patients in terms of disease severity and mechanisms by which their disability irreversibly worsens.¹ While in some patients mechanisms related to acute inflammation prevail, in other patients disability increases progressively. Acute inflammation, chronic inflammation and neuronal loss may all be present from onset in all patients, with varying proportions across individuals and over time. This high pathological heterogeneity may strongly influence the types of unfavourable outcomes to which individuals are more susceptible.¹

Over the past 10–15 years, the number of immunomodulatory treatments able to modify the disease course of MS has increased notably. However, not all of these drugs may necessarily impact the same types of outcome or their underlying mechanisms or have equivalent levels of effect. Therefore, timely prediction of a higher risk of developing significant disability may help to guide the choice of therapy and manage treatments.²

To date, prognostic models have focused mainly on predicting a single or limited number of outcomes.^3-5 In particular, many predictive models have focused on the risks of reaching Expanded Disability Status Scale (EDSS) milestones.^3,5 However, manifestations of clinical and/or radiological changes in MS are often more varied, including EDSS score changes, radiological activity or relapses.¹ It is therefore important to consider multiple predicted outcomes simultaneously.⁶ Furthermore, few prognostic models have included MRI data, such as the numbers of brain and spinal cord T2 lesions, which are known to play an important role in disease prognosis^3,7 and are easily obtainable in practice. Many proposed predictive models evaluate group associations, limiting their use in individuals,³ or lack external validation, limiting their generalizability.

Recently, we proposed a prognostic tool—the Barcelona-Baseline Risk Score (Barcelona-BRS) umbrella model—which supports long-term predictions of a wide range of outcomes at the time of the first demyelinating attack.⁸ However, these predictions can only be made at the group level, as the Barcelona-BRS employs a risk stratification approach based on a survival model for time to EDSS 3.0.⁸

To predict a wide range of relevant outcomes at the individual level, we buildt a statistical model that takes in account most of the data available in clinical practice, including treatment exposure. This prognostic framework, namely, the Spider-MS model, was built as a set of models, one for each outcome and has significant translational purpose. The model was validated using a separate external cohort to confirm its generalizability.

Materials and methods

Study design and participants

This study was based on a retrospective analysis of mostly prospectively collected data, following the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines (https://www.strobe-statement.org/). We used the Barcelona first-attack cohort^3,9 to build the Spider-MS model, which was externally validated in the Royal Melbourne Hospital Neuroimmunology Centre inception cohort.

Barcelona first-attack cohort

We included patients with a first demyelinating attack suggestive of MS seen at the Multiple Sclerosis Centre of Catalonia (Cemcat) between 1994 and October 2023, whose symptoms could not be explained by any other neurological condition and who were 50 years or younger at the time of the first attack. Eligible patients were clinically assessed and underwent a first brain MRI scan within 3 and 5 months of the first neurological symptoms suggestive of MS, respectively, and had undergone at least three EDSS assessments over the follow-up period. All data were prospectively collected from the first visit.

Royal Melbourne Hospital Neuroimmunology Centre inception cohort

We included patients who presented for the first time at the Neuroimmunology Centre, Royal Melbourne Hospital, between 2008 and 2015. Eligible patients were aged 18–50 years at symptom onset, had been seen in the clinic within 6 months of their first symptoms, including completing a diagnostic brain MRI, and had undergone at least two EDSS assessments.

Data collection

Demographic and clinical data

For all patients, we prospectively obtained information on demographic data: sex and age at the first demyelinating attack; and clinical data: date and topography of the first attack, presence and dates of relapses, level of disability according to the EDSS score at study baseline, i.e. within 3 months after the first demyelinating attack, and every 6–12 months afterwards. Information on disease-modifying treatment (DMT) exposure was also prospectively collected. DMTs were divided into high-efficacy drugs (monoclonals, cyclophosphamide, mitoxantrone) and moderate-efficacy drugs (the rest).

Brain and spinal cord MRI data

In the Barcelona cohort, all patients underwent brain MRI scans, including at least T2/fluid-attenuated inversion recovery (FLAIR)- and T1-weighted (both before and after administration of gadolinium-based contrast agents) sequences, using 1.5 or 3 T MRI scanners, at study baseline, i.e. within 5 months of symptom onset, 1 year later, and afterwards with a frequency between 6-monthly and 5-yearly, depending on clinical needs. Most patients also underwent spinal cord MRI scans during their routine follow-up assessment. However, spinal cord MRI at the first attack was only performed systematically in all patients after 2007. In the external validation cohort, imaging data were retrospectively acquired at the Royal Melbourne Hospital, using a 3 T (Siemens) field strength and standard diagnostic demyelinating protocol, including at least T2/FLAIR- and T1-weighted sequences (both before and after administration of gadolinium-based contrast agents). The MRI variables used for statistical modelling are explained later.

Laboratory data

For patients who underwent a lumbar puncture at their first attack, we obtained information about the presence of oligoclonal bands (OBs) in the CSF and serum. OBs were tested using agarose gel isoelectric focusing combined with immunoblotting, as previously reported.³ We defined ‘positive OBs’ whenever OBs were found in the CSF suggesting intrathecal OB production, i.e. absence of OBs in serum (type 2 pattern) or OBs in CSF and identical bands in serum, plus additional bands only in CSF (type 3 pattern). OB data were collected prospectively in both cohorts at the time of diagnosis.

Definition of outcomes

The Spider-MS model consisted of a series of eight predictive models, one for each of the following clinical outcomes. Inflammatory outcomes included: (i) McDonald 2017 criteria for MS¹⁰; (ii) a second clinical attack; and (iii) an annual rate of >2 new documented T2 lesions, a threshold that has been proven to be clinically relevant in studies on treatment response.¹¹ For this study, we focused on the first time this outcome was reached after the first attack. Clinical disability outcomes included: (i) confirmed and sustained disability worsening (CSDW), i.e. EDSS score worsening by at least 1.5, 1.0 or 0.5 steps if the baseline score was 0, 1.0–5.0 or >5.0, respectively, confirmed over 6 months and sustained over time. The last EDSS score—if recorded at least 12 months after disability worsening—was greater than or equal to the EDSS score that qualified for disability worsening; and (ii) a 6-month confirmed and sustained EDSS score of 3.0. Mechanism-driven disability outcomes included: (i) PIRA, defined as CSDW during a relapse-free period from 3 months after a relapse⁹ (or 6 months, in the case of the first demyelinating attack) to the next attack; (ii) relapse-associated worsening (RAW) at the first demyelinating attack: EDSS score ≥1.5 during the 12 months after the first attack, confirmed after 6 months and within the first year after the first attack (the confirmation score can fall within or outside a relapse-free period, as previously described⁸); and (iii) RAW at any other clinical relapse: a 6-month confirmed disability worsening that occurred within 3 months of a previous relapse (the confirmation score could fall within or outside a relapse-free period, as previously described⁸).

Statistical analysis

Descriptive statistics

We described the frequency of all study outcomes at the end of the follow-up and at 10-year follow-up (for those with a follow-up of at least 10 years or those who reached an outcome of interest before Year 10). Kaplan–Meier curves were used to visualize the development of each study outcome over time.

Spider-MS model

To build the Spider-MS model, for each of the outcomes mentioned earlier, we first designed Cox proportional hazards (PH) models, considering the following baseline variables as covariates:

1. demographics: age and sex;

2. clinical characteristics: topography of the first attack, i.e. optic nerve, spinal cord, brainstem (and cerebellum) and other;

3. brain MRI characteristics: categorized number of T2 lesions: 0, 1–3, 4–8, ≥9 or unknown; categorized contrast-enhancing lesions (CEL): 0, 1, >1 or unknown; and presence or absence of infratentorial lesions;

4. spinal cord MRI characteristics: categorized CEL: absent, present or unknown; categorized number of spinal cord lesions: 0, 1, 2–3, >3;

5. laboratory characteristics: presence of OBs in the CSF.

Furthermore, for all outcomes except for RAW at first attack, the model included, as covariates, the observed percentages of time exposed to high-efficacy and moderate-efficacy treatment between study baseline and the development of the study outcome (or between study baseline and the last follow-up visit). The reason for excluding the modelling of DMT effects for the RAW at first attack was the anticipated low number of patients receiving DMTs—especially high-efficacy DMTs—during the first year of the disease^8,9 and to avoid any form of immortal time bias.

We estimated 95% confidence intervals (95% CIs) of the hazard ratios (HRs) with 500 bootstrapped iterations (with replacement; percentile method) of each Cox model.

To optimize individualized predictions in external datasets, we built fully parametric bootstrapped (500 runs with replacement) accelerated failure survival models with Weibull distribution of time to each outcome, considering the same baseline variables as above. For each outcome, each patient was assigned a median of the logarithm of time, in years, to reaching the event. From these (log) median times, we calculated each patient’s probability of reaching each of the outcomes within 10 years from the first attack. To compute the percentage risk of developing the event at a given time t, given the set of covariates X, we followed this formula:

$$P(eventbytimet|X)=1-e^{-(\mathit{t}/e^{X\beta }{)}^{(1/scale)}}$$ (1)

$$Risk(\text{\%})=P(t|X)\times 100$$ (2)

Where: (i) $P(eventbytimet|X)$: probability of developing the event by time t, for a given person with a given set of covariates X (e.g. female, 32 years of age at symptom onset, etc.); (ii) X: set of covariate values; (iii) $X\beta$: set of covariates multiplied by the corresponding regression coefficients, which results in the risk score; (iv) scale: scale parameter, from the Weibull model; and (v) t: time at which we want to make the predictions (e.g. 10 years).

The accuracy of the Weibull models was assessed with the Harrell’s C-statistic. Then, for each patient, we developed a spider graph with all the outcomes of interest. Moreover the behaviour of the population for each outcome was included as a background spider image.

External validation

Predictive Weibull models from Spider-MS were applied to the Royal Melbourne Hospital Neuroimmunology Centre inception cohort using baseline data without recalibration to estimate individual risks of eight outcomes: McDonald 2017 criteria, second attack (CDMS), annual rate of new T2 lesions greater than two, RAW at the first attack, RAW at any subsequent relapse, CSDW (in those patients with at least three EDSS assessments) and PIRA and EDSS score ≥3.0. Model accuracy was assessed using the Harrell’s C-statistic.

Further, patients were classified into low-, medium- and high-risk groups according to the tertiles of the predicted risk of each of the outcomes. Kaplan–Meier survival curves were used to compare the observed proportions of each risk group who reached the outcomes over time.

Cox proportional hazards models (with Firth’s correction to account for groups with no observed events) were built to predict each of the observed outcomes conditional on the assigned risk group (based on the Spider-MS model). HRs (95% CI) were estimated to compare the high- and medium-risk groups with the low-risk group. The models were validated when the three Spider-MS-based risk groups were stratified correctly, in keeping with the original model.

Sensitivity analyses

Spider-MS developed using alternative approaches to modelling DMT exposure

We performed a first series of sensitivity analyses in relation to how we dealt with treatment exposure: (i) Spider-MS model with both moderate-efficacy and high-efficacy treatments considered together as DMT exposure, which might be seen as more convenient or generalizable in contexts where the type of exposure cannot be specified; and (ii) Spider-MS model where the effects of DMTs are not explicitly modelled, for reference.

Spider-MS developed in patients with abnormal MRI only

We performed another set of sensitivity analyses building the models on the original cohort but only including patients with an abnormal brain MRI at baseline. The aim of this sensitivity analysis was to assess whether the fact of including patients with normal brain MRI at baseline could have affected model estimations and accuracies.

Model calibration and discrimination at different time horizons

As a third set of sensitivity analyses, we evaluated model calibration at different time horizons, through assessing observed frequencies of event occurrence and predicted risks at 5-, 7.5- and 10-year follow-up. We also assessed model discrimination using Harrell’s C-index at these exact time horizons (on the original cohort), in addition to the overall follow-up (main analysis).

For these last sensitivity analyses, which implied reducing the sample size, we used a Spider-MS model which required a lower number of parameters to be estimated than the original one, i.e. a Spider-MS model which considered both DMT effects (moderate-efficacy and high-efficacy) together.

Results

Descriptive statistics

From a total of 1537 patients presenting with a first demyelinating attack at the Multiple Sclerosis Centre of Catalonia since 1994, we included 1180 patients, who fulfilled the inclusion criteria. The inclusion flow chart is shown in Fig. 1 and patient characteristics at baseline and over time are described in Table 1. Supplementary Figs 1 and 2 show the distribution of times from the first attack to first DMT exposure and first high-efficacy DMT exposure, in those exposed to DMTs and high-efficacy DMTs, respectively.

Figure 1.

Patient flow charts: original and validation cohorts. Patient flow chart for the original (Barcelona) (A) and validation (Melbourne) (B) cohorts. *153 patients were excluded from the Barcelona first-attack cohort due to: age >50 years at first attack [n = 4 (2.6%)]; alternative diagnosis [n = 84 (54.9%)]; did not agree to sign informed consent [n = 13 (8.5%)]; exceeded inclusion window [n = 27 (17.6%)]; or reported a previous relapse [n = 25 (16.3%)]. EDSS = Expanded Disability Status Scale.

Table 1.

Demographic and clinical characteristics at study baseline and over time

Baseline characteristics	Original cohort, N = 1180	Validation cohort, N = 108
Female sex, n (%)	810 (69%)	78 (72.2%)
Age at the first demyelinating attack in years, mean (SD)	32.37 (8.34)	32.41 (7.97)
First attack topography
Optic neuritis	405 (34%)	31 (28.7%)
Brainstem	290 (25%)	22 (20.4%)
Spinal cord	349 (30%)	30 (27.8%)
Other	136 (12%)	25 (23.1%)
No. brain T2 lesions at first attack (category)
0 lesions	264 (22%)	3 (2.8%)
1–3 lesions	153 (13%)	23 (21.3%)
4–8 lesions	142 (12%)	31 (28.7%)
>8 lesions	510 (43%)	51 (47.2%)
Unknown (brain MRI data not available)	111 (9.4%)	0 (0.0%)
No. brain contrast-enhancing lesions at first attack (category)
0 lesions	497 (42%)	45 (41.7%)
1–2 lesions	176 (15%)	22 (20.4%)
>2 lesions	105 (8.9%)	12 (11.1%)
Unknown (MRI with contrast not done/data not available)	402 (34%)	29 (26.9%)
No. spinal cord lesions at first attack (category)
0 lesions	415 (35%)	27 (25.0%)
1 lesion	126 (11%)	17 (15.7%)
2–3 lesions	62 (5.3%)	16 (14.8%)
>3 lesions	72 (6.1%)	7 (6.5%)
Unknown (spinal cord MRI not done/data not available)	505 (43%)	41 (38.0%)
Oligoclonal bands at first attack, n (%)
Negative	379 (32%)	10 (9.3%)
Positive	615 (52%)	28 (25.9%)
Unknown	186 (16%)	70 (64.8%)
Follow-up time in years, median (IQR)	10.8 (5.1, 18.0)	10.10 (8.08, 12.82)
Patients receiving DMTs at any time during the follow-up, n (%)	642/1180 (54.41%)	98/108 (90.7%)
Patients receiving high-efficacy DMTs at any time during the follow-up, n (%)	193/1180 (16.36%)	56/108 (51.85%)
Time to first DMT exposure in years,a median (IQR)	0.72 (0.41, 2.10)	0.46 (0.25, 0.87)
Time to first high-efficacy DMT in years,b median (IQR)	5.48 (1.03, 11.87)	3.77 (1.24, 5.13)
Outcomes occurring during the whole follow-up, n (%)
McDonald 2017 criteria for MS	797/1180 (67.5%)	98/108 (90.7%)
Second attack (CDMS)	563/1180 (47.7%)	73/108 (67.6%)
Annual rate of new T2 lesions >2	395/1180 (33.5%)	52/108 (48.1%)
RAW at the first attack	323/1180 (27.4%)	24/108 (22.2%)
RAW at subsequent attacks	121/1180 (10.3%)	18/108 (16.7%)
CSDW	384/1180 (32.5%)	33/108 (30.6%)
PIRA	290/1180 (24.6%)	12/108 (11.1%)
EDSS 3.0	195/1180 (16.5%)	20/108 (18.5%)

Patient characteristics at baseline and follow-up. CDMS = clinically defined MS; CSDW = confirmed and sustained disability worsening; DMT = disease-modifying treatment; EDSS = Expanded Disability Status Scale; IQR = interquartile range; MS = multiple sclerosis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening; SD = standard deviation; SPMS = secondary progressive MS.

^aRefers only to those patients who are exposed to DMTs; note that only n = 373 patients received DMTs during the first year of the disease in the original cohort.

^bRefers only to those patients who are exposed to high-efficacy DMTs; note that only n = 50 patients received high-efficacy DMTs during the first year of the disease in the original cohort.

Patients were followed up for a median time of 10.8 years [interquartile range (IQR) 5.1, 18.0 years]. Over the whole follow-up period, 563 (48%) patients experienced a second attack, 195 (17%) patients reached an EDSS of 3.0 and 52 (4.4%) reached an EDSS of 6.0. Furthermore, 290 (25%) developed at least one PIRA event. Supplementary Fig. 3 shows the Kaplan–Meier survival curves for all study outcomes. Figure 2 provides a polyhedral description of all observed study outcome frequencies.

Figure 2.

Polyhedral visualizations of observed outcomes and 10-year predictions from the Spider-MS model. Original and external validation cohorts. (A) Observed proportion of patients who reached study outcomes in the original cohort over the whole follow-up (n = 1180); (B) observed proportion of patients who reached outcomes in the original cohort in those patients with ≥10 years of disease duration (n = 642); (C) predicted percentage risks at 10-year follow-up based on the Spider-MS model, in the original cohort (n = 1180); (D) individualized prediction of study outcomes at 10-year follow-up based on the Spider-MS model, for a (real) 46-year-old female with a first demyelinating attack from the original cohort; (E) individualized prediction of study outcomes at 10-year follow-up based on the Spider-MS model, for a (real) 27-year-old female with a first demyelinating attack from the original cohort; (F) small descriptive table summarizing patient characteristics. (G) Observed proportion of patients who reached study outcomes in the validation cohort over the whole follow-up (n = 108); (H) observed proportion of patients who reached study outcomes in the validation cohort in those patients with ≥10 years of disease duration (n = 55); (I) predicted percentage risks at 10-year follow-up based on the SPIDER-MS model, in the validation cohort (n = 108); (J) individualized prediction of study outcomes at 10-year follow-up based on the Spider-MS model, for a (real) 38-year-old female with a first demyelinating attack from the validation cohort; (K) individualized prediction of study outcomes at 10-year follow-up based on the Spider-MS model, for a (real) 27-year-old female with a first demyelinating attack from the validation cohort; (L) small descriptive table summarizing patient characteristics. Please note that both patients from the original cohort only differ in age at first attack. Please also note that both patients from the validation cohort only differ in age at first attack. CDMS = clinically defined MS; CSDW = confirmed and sustained disability worsening; EDSS = Expanded Disability Status Scale; MS = multiple sclerosis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening.

Spider-MS model

According to the Cox PH models initially built to predict study outcomes (Fig. 3 and Supplementary Table 1), an older age at first attack was associated with a greater risk of developing RAW at first attack [HR = 1.268 (95% CI 1.086, 1.494), P < 0.01] and PIRA [HR = 1.345 (1.116, 1.626), P < 0.001]. However, it was also significantly associated with a lower risk of meeting the McDonald diagnostic criteria [HR = 0.843 (0.765, 0.929), P < 0.001], a clinical relapse after the first attack (or CDMS) [HR = 0.791 (0.695, 0.902), P < 0.001], an annual rate of new T2 lesions greater than two [HR = 0.606 (0.503, 0.719), P < 0.001] and RAW at clinical relapses after the first attack [HR = 0.646 (0.475, 0.853), P < 0.001].

Figure 3.

Cox proportional hazards model estimates for the different study outcomes. The predictor variables are the same for all models and appear in the following order: demographic characteristics (magenta): age at first attack, sex (male versus female); first attack topography (red): brainstem (versus ON), SC (versus ON), other topography (versus ON); number of brain lesions (categorical, orange): 1–3 lesions (versus 0), 4–8 lesions (versus 0), >8 lesions (versus 0), unknown (versus 0); number of infratentorial lesions (categorical, in yellow): >0 lesions (versus 0), unknown (versus 0); number of brain CELs (categorical, green): 1–2 CELs (versus 0), >2 (versus 0), unknown (versus 0); spinal cord lesions (categorical, light blue): 1 SC lesion (versus 0), 2–3 SC lesions (versus 0), >3 SC lesions (versus 0), unknown (versus 0); number of SC CELs (categorical, blue): >0 SC CELs (versus 0), unknown (versus 0); CSF OBs (dark blue): positive (versus negative), unknown (versus negative); percentage time on moderate-efficacy DMT before the outcome (continuous variable, purple); percentage time on high-efficacy DMT before the outcome (continuous variable, purple). Note that a small tilted square is place at the end of the 95% CI whenever the limits of that 95% CI exceeded the graph area, for convenience. Also note that the model for RAW at the first attack did not include DMT exposure as a predictor (‘Materials and methods’ section for more details). For all outcomes except for PIRA, the number of brain lesions was the most important predictor. Refer to the main text and Supplementary Table 1 for full details on model estimates. CI = confidence interval; CDMS = clinically defined multiple sclerosis (2nd demyelinating attack); CELs = contrast-enhancing lesions; CSDW = confirmed and sustained disability worsening; DMT = disease-modifying treatment; EDSS = Expanded Disability Status Scale; HRs = hazard ratios; MS = multiple sclerosis; OBs = oligoclonal bands; ON = optic neuritis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening; SC = spinal cord.

A first attack involving the spinal cord was strongly associated with a greater risk of RAW at the first attack [HR = 1.922 (1.355, 2.765), P < 0.001].

In general, a greater number of T2 lesions and a greater number of gadolinium-enhancing lesions at baseline, in the brain and the spinal cord MRI, were associated with a greater risk of developing any of the study outcomes. These associations were stronger for the brain than for the spinal cord lesion numbers (Fig. 3 and Supplementary Table 1). The presence of more than three lesions within the spinal cord was only significantly associated with developing a second attack of MS (CDMS) [HR = 2.155 (1.187, 3.582), P < 0.001].

In general, the presence of OBs in the CSF was associated with greater risks of unfavourable outcomes, although it only reached the level of evidence for McDonald MS [HR = 2.378 (1.881, 3.017), P < 0.001], developing CDMS [HR = 1.556 (1.177, 2.08), P < 0.01], RAW at subsequent attacks [2.064 (1.079, 4.475), P < 0.05] and CSDW [HR = 1.515 (1.103, 2.148), P < 0.05].

Finally, for all outcomes except for RAW at the first attack, where DMT effects were not assessed, greater percentages of time on high-efficacy DMTs and, to a lesser extent, moderate-efficacy DMTs before a given outcome were associated with lower risks of reaching that outcome (Figs 3–5). However, these associations did not reach the level of statistical evidence for PIRA (either moderate-efficacy or high-efficacy DMTs). Furthermore, for some outcomes (RAW at subsequent attacks and CSDW), only the high-efficacy DMT exposure reached the level of evidence.

Figure 5.

Strongest predictors of study outcome based on the Spider-MS model: conceptual figure. Conceptual figure (‘galaxy plot’) illustrating the effects of baseline predictors and treatment exposure on the risk of study outcomes. Circle size represents the size of the association—note this is a schematic figure. Protective effects are displayed within the inner circle. Only the strongest associations are displayed (P < 0.05, with solid shape outlines; and P = 0.05–0.10, with dashed shape outlines). As observed, brain T2 lesion load is, by far, the strongest predictor of all outcomes except for PIRA, where the effect of higher numbers of brain lesions is very small, comparable to or even smaller than older age. For all outcomes except for PIRA, greater DMT exposure, and especially high-efficacy DMT, represented as 80% units of time on DMT before the outcome to allow a better visualization, implied lower risks of reaching the outcome. However, the effects of DMT exposure were very different across outcomes, being highest for the most inflammatory ones, i.e. fulfilling McDonald MS diagnosis criteria, developing a second attack, or reaching an annualized rate of new T2 lesions greater than two. Please note that the effects of age are displayed for each decade older at the time of the first attack, also to allow a better visualization. CI = confidence interval; CDMS = clinically defined multiple sclerosis (2nd demyelinating attack); CELs = contrast-enhancing lesions; CSDW = confirmed and sustained disability worsening; DMT = disease-modifying treatment; EDSS = Expanded Disability Status Scale; HRs = hazard ratios; MS = multiple sclerosis; OBs = oligoclonal bands; ON = optic neuritis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening; SC = spinal cord.

Effects of high-efficacy DMT exposure (for each percentage unit increase in time of DMT exposure) were greatest for CDMS [HR = 0.96 (0.647, 0.976), P < 0.001], annual rate of new T2 lesions >2 [HR = 0.968 (0.949, 0.977), P < 0.001], McDonald MS [HR = 0.972 (0.915, 0.991), P < 0.05] and EDSS 3.0 [HR = 0.973 (0.941, 0.987), P < 0.01].

All Cox PH estimates (adjusted HRs) are shown in Fig. 3 and Supplementary Table 1. Figure 5 is a conceptual diagram that summarizes the strongest predictors for each outcome.

The Spider-MS Weibull models to predict study outcomes achieved Harrell’s C values between 0.653 (to predict PIRA) and 0.823 (to predict an annual rate of new T2 lesions >2), indicating moderate to high accuracy (Table 2). The Spider-MS Weibull coefficients, which were then used in the validation cohort, confirmed the associations found in the Cox PH models. These coefficients are displayed in Supplementary Table 2.

Table 2.

Assessment of Spider-MS models’ accuracy in the original and external validation cohorts

Spider-MS	Harrell’s C (IQR)a (original cohort, HE and ME DMTs separately considered)b	Harrell’s C (95% CI)c (validation cohort, HE and ME DMTs separately considered)	Spider-MS risk groupsd (n = 36 per group)	HR (95% CI)e (validation cohort, HE and ME DMTs separately considered)	P-value
McDonald 2017 criteria for MS	0.819 (0.813, 0.823)	0.664 (0.605, 0.724)	Low (reference category)	–	–
			Medium	1.291 (0.783–2.130)	0.3
			High	2.787 (1.706–4.554)	<0.0001
Second attack (CDMS)	0.773 (0.767, 0.780)	0.686 (0.624, 0.750)	Low (reference category)	–	–
			Medium	1.796 (0.957–3.372)	0.06
			High	4.365 (2.392–7.966)	<0.0001
Annual rate of new T2 lesions greater than 2	0.823 (0.816, 0.830)	0.679 (0.607, 0.752)	Low (reference category)	–	-
			Medium	2.339 (1.041–5.254)	0.04
			High	4.335 (2.023–9.289)	<0.0001
RAW at the first attackb	0.676 (0.665, 0.687)	0.452 (0.307, 0.597)	Low (reference category)	–	–
			Medium	0.627 (0.217–1.811)	0.4
			High	1.369 (0.536–3.497)	0.6
RAW at subsequent attacks	0.771 (0.753, 0.784)	0.419 (0.261, 0.578)	Low (reference category)	–	–
			Medium	0.168 (0.037–0.766)	0.009
			High	0.542 (0.197–1.493)	0.2
CSDW	0.679 (0.668, 0.689)	0.623 (0.529, 0.717)	Low (reference category)	–	–
			Medium	1.894 (0.745–4.817)	0.2
			High	2.080 (0.839–5.161)	0.1
PIRA	0.653 (0.640, 0.665)	0.766 (0.650, 0.882)	Low (reference category)	–	–
			Medium	1.671 (0.151–18.44)	0.7
			High	9.113 (1.154–71.95)	0.01
EDSS 3.0	0.749 (0.735, 0.763)	0.729 (0.613, 0.846)	Low (reference category)	–	–
			Medium	1.050 (0.262–4.205)	0.999
			High	3.991 (1.278–12.468)	0.01

CI = confidence interval; CDMS = clinically defined MS; CSDW = confirmed and sustained disability worsening; DMTs = disease-modifying treatment; EDSS = Expanded Disability Status Scale; HE = high efficacy; IQR = interquartile range; ME = moderate efficacy; MS = multiple sclerosis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening; SE = standard error.

^aGiven that the Weibull models conforming Spider-MS have been created using 500 bootstrap samples, for each outcome there are 500 Harrell’s C values, of which we show the median (IQR).

^bAll models except that for RAW at the first attack take into account ME and HE DMTs separately; the model for RAW at first attack does not take into account DMT effects (see Methods for more details).

^cFor the external validation cohort, since we are applying an already estimated Spider-MS model, we provide the Harrell’s C value (95% CI) corresponding to each outcome [since we are not re-estimating the model, note that the 95% CI of the Harrell's C has been estimated assuming normality, based on the SE of the estimated C-index, as: C-index −1.96×SE, C-index +1.96×SE, and not through bootstrapping].

^dSpider-MS risk groups have been created through splitting the validation cohort into tertiles based on estimated individual risks for each one of the outcomes after applying the Spider-MS model (see Methods for more details).

^eHazard ratios (95% CI) and P-values for each Spider-MS risk group with respect to the low-risk group, for each one of the study outcomes.

Polyhedral prediction from the Spider-MS model

The Spider-MS model-based 10-year predictions of study outcomes are displayed as a polyhedral in Fig. 2, as well as numerically in Supplementary Table 3. Such predictions were very close to observed frequencies of study outcomes over the whole follow-up, considering all patients (n = 1180) and also those who had 10 years or more of disease duration (n = 642). Furthermore, the Spider-MS allowed individual patient predictions, as also shown in Figs 2 and 4.

Figure 4.

Simulated scenarios based on the Spider-MS model. This figure displays 10 different scenarios according to DMT exposure for one imaginary patient: a 38-year-old male, who is exposed to either moderate efficacy DMTs (A–E) or high-efficacy DMTs (F–J). For this imaginary patient, the following characteristics were defined: spinal cord topography of the first attack, 4–8 brain lesions, 1–2 brain contrast-enhancing lesions, at least 1 infratentorial lesion, 1 spinal cord lesion, at least 1 spinal cord contrast-enhancing lesion, positive OBs. ME or HE DMT exposure in this figure ranges between 1% of the time (A and F) and 99% of the time (E and J). This figure shows how the increasing exposure to DMTs is associated with lower risks of developing outcomes, especially inflammatory outcomes. Furthermore, the estimated treatment effects seem to be much higher for HE DMTs than for ME DMTs. Notably, though, none of the two DMT types seems to have a definitive effect on PIRA. CI = confidence interval; CDMS = clinically defined multiple sclerosis (2nd demyelinating attack); CELs = contrast-enhancing lesions; CSDW = confirmed and sustained disability worsening; DMT = disease-modifying treatment; EDSS = Expanded Disability Status Scale; HE = high-efficacy; HRs = hazard ratios; ME = moderate-efficacy; MS = multiple sclerosis; OBs = oligoclonal bands; ON = optic neuritis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening; SC = spinal cord.

External validation

We included 108 patients with a median follow-up of 10 years. Table 1 shows the main clinical and demographic characteristics at baseline.

The median age at first attack was 32.4 years [standard deviation (SD) 7.97], and 78 (72.2%) were female. Patients were followed up for a median time of 10.10 years (IQR 8.08, 12.82). Over the whole follow-up period, 73 (67.6%) patients had a second attack, 20 (18.5%) patients reached an EDSS of 3.0 and 12 (11.1%) developed at least one PIRA event. Figure 2 provides a polyhedral description of the proportion of patients who reached the study outcomes, also described in Table 1.

Without any recalibration, the Spider-MS model was applied to the external validation cohort and individual predictions were obtained for each patient and each outcome. Furthermore, individual patient percentage risks for each outcome at 10-year follow-up were estimated.

Except for models predicting RAW at the first attack and RAW at subsequent attacks, Spider-MS models were, in general, well validated, with Harrell’s C-statistics from the Spider-MS Weibull models ranging from 0.77 (PIRA) to 0.62 (CSDW), as detailed in Table 2. For RAW at the first attack and RAW at subsequent attacks, the Harrell’s C-statistics were 0.452 (0.307, 0.597) and 0.419 (0.261, 0.578), respectively, and were considered as non-validated.

The mean percentage risks across all patients were calculated and displayed as a polyhedral plot (Fig. 2). These percentage risks closely resembled those estimated in the original cohort.

Patients were then classified into low-, medium- and high-risk groups, based on the individual predictions derived from the Spider-MS model (developed in the original cohort), for each outcome (‘Materials and methods’ section). The Kaplan–Meier curves showed that, for all outcomes except for RAW at subsequent relapses, those patients who had been classified within the high-risk group based on the Spider-MS model developed the outcome earlier than those patients classified as being at low risk (Fig. 6). Log rank tests showed significant differences across groups for McDonald MS (P < 0.001), CDMS (P < 0.001), annual rate of new T2 lesions greater than two (P < 0.001), PIRA (P = 0.01) and EDSS 3.0 (P = 0.01) (Fig. 6). These differences across groups were then confirmed by Cox models, as shown in Table 2.

Figure 6.

Kaplan–Meier survival curves of Spider-MS risk groups in the validation cohort. Kaplan–Meier curves built to assess the accuracy of the Spider-MS model in the external validation cohort. Please note that, in all models except for that of RAW at first attack, moderate-efficacy and high-efficacy DMTs were separately considered. For RAW at first attack, the model did not include DMT exposure. For most outcomes (all except for the RAW at first and subsequent attacks), those patients in the validation cohort that were classified as high risk based on the Spider-MS model had a higher observed rate of disease outcomes than patients in the low-risk group, which was mirrored by significant log rank tests results. CDMS = clinically defined MS; CSDW = confirmed and sustained disability worsening; DMT = disease-modifying treatment; EDSS = Expanded Disability Status Scale; MS = multiple sclerosis; PIRA = progression independent of relapse activity; RAW = relapse-associated worsening.

Sensitivity analyses

Spider-MS developed using alternative approaches to modelling DMT exposure

We first built a Spider-MS model which used total DMT exposure, i.e. without separating moderate- and high-efficacy drugs. The predictions for each of the outcomes within this Spider-MS model showed very similar results to those obtained with the main Spider-MS model, although the accuracies were, in general, slightly lower (Supplementary Tables 3–5). Supplementary Fig. 4 displays the Cox models’ coefficients plots and Supplementary Fig. 5 displays the model in a conceptual manner, showing the strongest predictors.

We then built a Spider-MS model without explicitly modelling DMT exposure. Again, the overall predictions were similar to those predictions from models which explicitly took into account DMT effects, although with lower accuracy, especially for CDMS and annualized new T2 lesion rate >2 (Supplementary Tables 6–8). Supplementary Fig. 6 displays Cox model coefficients of this Spider-MS without explicitly modelling DMTs. Supplementary Fig. 7 displays this model without accounting for DMTs in a conceptual manner (strongest predictors).

These two additional sets of Spider-MS models have also been validated in the external validation cohort, as the main Spider-MS model, with similar results to those obtained with the validation of the main Spider-MS model (Supplementary Table 6).

Spider-MS developed in patients with abnormal MRI only

When the models were built only on patients with an abnormal brain MRI at baseline (Supplementary Table 9 shows the main characteristics of the restricted cohort), the accuracies of the models were similar to those obtained with the full dataset (Supplementary Table 10).

Model calibration and discrimination at different time horizons

When observed and predicted risks of event occurrence were assessed at 5-, 7.5- and 10-year follow-up (Supplementary Table 11), we could observe that the actual values were very similar. Notably, though, Harrell’s C index values were higher as the time horizon moved closer to study baseline (Supplementary Table 12).

Discussion

In this study, we have built the first multimodal statistical framework to predict multiple long-term outcomes at a first demyelinating event. This large model, named Spider-MS, has revealed important associations between baseline clinical and demographic characteristics of patients presenting with the first attack of MS and the risk of future unfavourable outcomes, as well as the association between DMT and better prognosis. Of note, with the Spider-MS model we provide weights (i.e. model coefficients) that have proved largely generalizable, as shown by the external validation analysis. The Spider-MS model allows individualized comprehensive (multi-outcome) predictions to be visualized in individual polyhedral plots, allowing comparisons among subjects and across cohorts. For all these reasons, we consider Spider-MS to be a helpful and promising predictive tool that may enhance patient management in the MS clinic.

Predictive models so far

Over the last few years, a great effort has been made to predict clinical outcomes in MS. This prediction has been mainly explored through building survival models based on clinical data and, more recently, on MRI data.^3,12 These models, when built using data at the time of the first demyelinating attack, have shown a unique ability to discriminate which patients are at the highest risk of progression. In particular, our recently proposed Barcelona-BRS model has demonstrated a high ability to classify patients into prognostic groups, with distinct risks of unfavourable outcomes in the long term.⁸ However, despite its power and generalizability, the Barcelona-BRS model had a key limitation: the risk groups were defined based solely on the risk of reaching EDSS 3.0. Interestingly, this initial approach to risk stratification proved sufficient to show that patients classified in the highest-risk BRS group were also more likely to develop other unfavourable outcomes—ranging from predominantly inflammatory to predominantly neurodegenerative—beyond the specific disability milestone (EDSS 3.0), for which the model had been originally designed. While this could be seen as an advantage of the BRS model when used as a general-risk prediction tool, the wide range of disease milestones and the growing array of disease-modifying treatments in MS—each targeting distinct mechanisms of action (and possibly affecting distinct milestones as well)—make the ability to predict specific outcomes an essential requirement for any prediction tool to be clinically useful. In that regard, our proposed approach, the Spider-MS predictive model, allows outcome-specific predictions. Furthermore, it also provides individualized prognostications. To this end, an individual patient is assigned a predicted probability of experiencing each of the modelled outcomes within a given time horizon. The result is displayed in an intuitive manner (i.e. in a radar plot, emulating a spider net). The Spider-MS predictive model also considers DMT effects and here we provide all of its model coefficients (Supplementary material) for them to be applied in future, external cohorts.

Baseline predictors of long-term outcome-specific risks

The Spider-MS models confirmed important associations between known predictors and clinical outcomes, which are displayed in a schematic, conceptual way in Fig. 5. For instance, we found a strong association between greater T2 lesion load and worse outcomes.^3,12-14 Nonetheless, the effects of greater T2 lesion load did not seem equal for all outcomes: while the HRs (95% CI) for the highest brain lesion category (>8 lesions) for reaching McDonald MS or an annual rate of new T2 lesions greater than two were 17.79 (95% CI 11.99, 27.51) and 26.71 (12.66, 92.78), respectively, the equivalent HRs for predicting PIRA or CSDW were 1.60 (0.91, 2.79) or 5.65 (2.57, 18.67), suggesting a much smaller effect, in line with the more neurodegenerative nature of the latter outcomes.^15,16 These findings were supported by the negative association between the presence of contrast-enhancing lesions on brain MRI and CSDW and PIRA (although the latter did not reach the statistical level of evidence). Interestingly, a recent systematic review about prediction of long-term outcome in MS, which analysed the results of 241 articles (40 of them specifically focused on predictive models), has also highlighted the importance of high lesion load at symptom onset as a major predictor of poor long-term prognosis.¹²

The role of age is of particular interest: while an older age at symptom onset was associated with greater risk of PIRA, EDSS 3.0 and RAW at first attack, it showed a protective effect for McDonald MS and reaching an annual rate of new T2 lesions greater than two. These findings underscore the major influence of older age on neurodegenerative outcomes, as previously described,¹² potentially reflecting diminished remyelination capacity with age,¹⁷ as well as the effects of immunosenescence, epigenetic changes and comorbidities.¹⁸ In addition, our results confirm the strong association between younger age and more inflammatory events, as also shown in the literature.¹⁹ The role of older age in RAW at subsequent attacks is noteworthy, since in our model it appeared to have a protective effect. This may indicate that, at least in our cohort, this outcome—characterized by ‘mixed’ mechanisms—was predominantly influenced by underlying acute inflammatory activity. Future studies focusing on the mechanisms and predictors of RAW, both at first and subsequent attacks, are therefore warranted.

Our model also revealed the effects of other, more modest predictors of unfavourable outcomes, such as the presence of high spinal cord T2 lesion load (>3 lesions), contrast-enhancing lesions in the brain or in the cord, or the presence of OBs in the CSF, which were mainly associated with eminently inflammatory outcomes, i.e. fulfilling the 2017 McDonald diagnostic criteria of MS, a second demyelinating event, an annual rate of new brain T2 lesions greater than two, or to a lesser extent, RAW. The more modest role played by the presence of CSF OBs in relation to the other predictors, such as T2 lesion load, has already been described in the literature.³ Similarly, the limited role played by gadolinium-enhancing lesions at symptom onset has also been observed in other studies.¹² A bit less expected were the relatively moderate associations for the spinal cord lesions compared with the brain lesions, especially for the measures of progression, according to previous research.^7,20 Nonetheless, to predict RAW, especially RAW at the first attack, a spinal cord topography at the clinically isolated syndrome seemed to be particularly relevant, in line with the literature linking spinal cord lesions and higher disability levels in the short term.²¹ Reasons behind the lower-than-expected relevance of spinal cord lesions in our study may be related to the fact that we did not include spinal cord MRI at the first attack in our routine practice until 2007. Therefore, many of our patients had to be assigned a missing value in that field for the construction of our models, limiting our ability to assess the true value of cord lesions as predictors. Furthermore, the fact that only the most recent patients had been assessed on spinal cord lesions meant, in practice, shorter follow-up times to evaluate the predictive value of such cord lesions as compared with other predictors which have been systematically collected since 1994. Importantly, this temporal bias in spinal cord missing data also meant that patients with spinal cord lesions assessed were more likely to commence high-efficacy DMTs, since these have only been broadly available over the last 10–15 years.²² This implies that the detrimental effects attributable to cord lesions were probably mitigated by the effect of highly efficacious treatments, according to the unequivocal evidence supporting the power of DMTs and especially high-efficacy DMTs to change the disease course of MS.²³ Finally, we should take into account the technical challenges related to the detection of cord lesions, which might have obviated a thorough assessment of their prognostic implications.²⁴

As a sensitivity analysis, we restricted the cohort to patients with abnormal MRI at first visit. Model performance changed minimally, supporting the robustness of including all patients with a first demyelinating attack regardless of baseline lesion burden.

Furthermore, we explored the ability of the models to predict outcomes at earlier time points, exploring their discrimination capabilities and calibration. We found that, although predicted and observed probabilities of unfavourable outcome occurrence were similar across time horizons, the discrimination ability of the models seemed slightly greater at earlier time points. This finding was expected, given the dynamic nature of the disease. That is, certain events that occur after disease onset (i.e. development of new T2 lesions, the development of PIRA or RAW, poor response to treatment, etc.) cannot be accurately predicted at baseline, but can influence the future disease trajectory. This phenomenon, also seen in artificial intelligence-based predictions,²⁵ warrants development of dynamic models of disease prognosis in MS.

It is important to note the strong association between a greater percentage of time on DMT before a given outcome and a longer time from symptom onset to that outcome, especially for the most inflammatory outcomes and in relation to high-efficacy DMT exposure. This suggests that DMTs play a protective role in relation to most studied outcomes. Given the mechanisms of action of all MS drugs approved so far, our findings were not surprising for the most inflammatory outcomes.²² However, their association with less inflammatory or mixed inflammatory-neurodegenerative outcomes, including CSDW, EDSS 3.0 and, to a lesser extent, RAW at subsequent attacks, was of particular interest.^26,27 This is in keeping with the previous reports of associations of the available DMTs with a lower risk of progression.^28,29 However, in our study, although DMT exposure to either moderate-efficacy or high-efficacy therapies seemed to imply a lower risk of developing PIRA, in line with previous studies,³⁰ no significant associations were observed. Indeed, as has been proposed in the literature, while PIRA might be partly driven by acute inflammation,^9,31 it is possibly the presence of chronic inflammation and neurodegeneration that contributes the most to its development.^15,32-36 In the future, our model may be very useful to assess the ability of new treatments—such as Bruton Tyrosine Kinase inhibitors—to target neurodegenerative outcomes such as PIRA, which are not clearly prevented by current high-efficacy drugs. It is also important to note the marginal effect of DMT exposure on RAW at subsequent attacks, which is also limited to the effects of high-efficacy DMTs. This finding highlights, on one hand, the role of acute inflammation on the development of relapses and the clear protective effects of high-efficacy therapies in that regard. Yet it especially emphasizes the likely, strong effect of neurodegenerative mechanisms underlying RAW, which like PIRA, are not effectively countered by currently available drugs. Finally, it is worth mentioning that our model was not able to assess DMT effects on RAW at the first attack, partly due to the limited number of patients receiving DMTs (n = 373) and especially, high-efficacy DMTs (n = 50) during the first year of the disease in our development cohort. Additionally, it was also partly due to the high risk of immortal time bias. That is, all patients exposed to DMTs received them after a certain minimum time, which is almost impossible to shorten in clinical practice because it is inevitably related to diagnostic work up and administrative processes. Notably, although this risk of immortal time bias might have existed for other outcomes too, we believe it is relatively minor due to the long times needed to reach such different outcomes in relation to the actual immortal time at disease onset. Furthermore, the fact that treatment effects were so different across outcomes also probably indicates that the models are indeed capturing some ‘true’ DMT effects.

These findings are displayed in Fig. 4, where the risks of one imaginary patient are shown under 10 scenarios differing only on the percentage of time exposed to moderate-efficacy or high-efficacy DMTs, again showing the important effect of high-efficacy therapies on the most inflammatory outcomes. Figure 5 also displays the estimated effects of DMT exposure (among the other significant associations) on the risks of reaching the study outcomes, clearly showing where DMTs might have the strongest impact.

Reassuringly, the models built without explicitly considering DMT effects showed similar signals to those seen in the model with DMT effects. However, despite the similar accuracy (Harrell’s C) values of both Spider-MS models, the strength of the associations was lower when DMT effects were not accounted for, as shown in Supplementary Figs 5 and 6. This supports the use of the most complex model including the stratified DMTs. Because the model not accounting for DMT still included several known clinical and radiological prognostic markers, this version of the Spider-MS model is useful as a reference model (Supplementary Tables 7 and 8), and might be preferred in contexts where it is not possible to foresee the type or length of future treatment, especially at the population level.

Of note, the ultimate aim of the Spider-MS model is to become a useful tool for clinical practice, to be applied to patients at symptom onset. However, clinicians will (obviously) not have the information on treatment exposure beforehand. Therefore, this DMT information will be the only variable of the model that will not depend on patient characteristics at baseline. Instead, this variable will be used by the clinician to demonstrate, in a particular patient, how the DMT exposure may change the risk of developing unfavourable outcomes over a given period (as shown in Fig. 4). Nonetheless, we acknowledge that the use of the Spider-MS as a decision-making tool in clinical practice will require additional work to independently validate its predictive ability in individuals in a prospective manner. The definitions of predictive and adjustment variables and design of the multivariable models may considerably influence accuracy of the resulting prognostic models. We have therefore included not only one but three sets of regression coefficients, reflecting different approaches to handling treatment exposure. The convergence of the estimates from these models provides reassurance in relation to the robustness of the Spider-MS model.

Polyhedral visualizations of risk predictions

One of the most appealing aspects of our Spider-MS model is the visualization of its predictions, which allows, at a glance, comparisons of the predicted outcomes across populations and between individual patients. For instance, we saw that the effect of age was very similar for the original and the external validation cohorts. Of note, though, the importance of visualizing the results in a user-friendly manner is not new. Several authors have provided risk visualization tools for people with MS. Even though most of these models were based on clinical data at the first attack,^37-40 some allowed use of longitudinal data over time.⁴¹ Future directions will include the development of a freely accessible web/app-based tool based on the Spider-MS model, aimed at supporting clinicians in patient management. Importantly, such a tool will also facilitate the essential step of evaluating the clinical utility of our model.

External validation

The generalizability of the Spider-MS model was supported by validation in an external cohort. First, we compared the observed outcomes with the predictions generated by the Spider-MS model in the external cohort, for each specific outcome. We did this by dividing the validation cohort into three equally sized groups based on tertiles of the predicted risk according to our model. For most outcomes—except the RAW at subsequent relapses, where we considered that the model could not be validated—patients predicted to be at the greatest risk developed the respective outcomes earlier than those patients classified as being at the lowest risk. For RAW at the first attack, patients at the greatest risk developed the event earlier than those at the lowest risk, but the group with predicted intermediate risk showed the best outcomes of all three groups. Therefore, we also consider the model for RAW at first attack as un-validated. For those outcomes strongly associated with acute inflammation, such as CDMS, annual rate of new T2 lesions greater than two, or fulfilling the 2017 McDonald criteria, the Spider-MS model showed a good performance upon validation. For more progressive outcomes, such as PIRA or EDSS 3.0, the model was well validated for identifying the high-risk group but was less effective at separating the low- and medium-risk groups. It should be noted that the two outcomes where the model could not be validated—RAW at the first attack and at subsequent attacks—were most likely affected by both acute inflammation and neuronal loss.

The external validation of the additional models—unstratified DMTs and without DMTs, whose results were very similar after the validation of the main Spider-MS model, provides further reassurance in the external validity of the models, in addition to the model convergence described above.

All these findings suggest that predicting complex composite outcomes driven by ‘mixed’ pathogenic mechanisms—and by predominantly neurodegenerative mechanisms too—is difficult based solely on demographic, clinical and radiological data, and remains a challenge. While this does not invalidate our model, it reminds us of the need for future research tackling these more complex outcomes, possibly through gaining a deeper understanding of the mechanisms behind them, refining the clinical definitions used for their identification,⁴² and integrating more specific biomarkers, such as serum neurofilament light chain (sNfL) and glial fibrillary acidic protein (sGFAP) levels,⁴³ as predictors.

Strengths, limitations and future directions

The main strengths of our work are the large and deeply phenotyped original and validation cohorts, both with a long clinical and MRI follow-up. The good results of the external validation analyses in general terms, except for RAW at the first attack and RAW at subsequent attacks—support the generalizability of the Spider-MS model.

There are also several limitations. First, many of the outcomes of the study are based on clinical definitions that may be changing over time.^42,44 This means that even after the Spider-MS model is implemented in practice, it will need to be regularly updated with new data and definitions. Another limitation is that, given the real-world nature of the study, the frequency of EDSS assessments was very variable across patients. This may have affected the reliability of the event definitions in our real-world cohorts and increased the heterogeneity in identifying some of the outcomes, such as PIRA and CSDW.^9,44 It is difficult to foresee the real impact of such heterogeneous clinical assessments on our model’s performance. However, it is likely that more regular EDSS assessments allowed us to identify patients with PIRA and CSDW with higher accuracy, aiding the accuracy of the predictive models. Additionally, the MRI data used as predictors in our model were only semi-quantitative, and the spinal cord and CSF OB data were very often missing. However, this is consistent with standard clinical practice, thus ensuring that the model is relevant to practising neurologists. Although we included an ‘unknown’ category in these variables, the model estimates (Fig. 3) suggest that the category ‘unknown’ was often informative. Future research will aim to minimize the rates of missing data to improve the predictive value of the Spider-MS model. Similarly, future research should aim to account for brain or cord atrophy measures as predictors—possibly together with features related to chronic active lesions (e.g. paramagnetic rim lesions),^45,46 which are known to have a strong impact on disability progression in multiple sclerosis and which were not included in our study.^12,24,46,47 Finally, the Spider-MS model was based on static predictions made at study baseline, limiting its ability to incorporate clinical and treatment-related events occurring later in the disease, which are known to be important for modelling disease progression in MS. Future research should expand Spider-MS into a dynamic tool.

Conclusions

We provide a promising tool for predicting long-term unfavourable outcomes in clinical practice, at the time of first presentation of MS, supported by robust findings in the original cohort and an external validation analysis. Pending further refinement through prospective validation in real-world settings, this tool, named Spider-MS, may have the potential to contribute to improving patient management in clinical practice and, consequently, improve the long-term prognosis.

Data availability

Data will be made available upon reasonable request to the corresponding and senior authors.

Funding

This research has been possible thanks to a Miguel Servet Contract awarded to C.T. by the Instituto de Salud Carlos III (ISCIII) and co-funded by the European Union in 2023 (CP23/00117), which has paid C.T.’s salary since 2024. Additionally, this research has also been possible thanks to research grants (PI21/01860, PI24/01277) awarded to C.T. by the Instituto de Salud Carlos III (ISCIII) and co-funded by the European Union, and another grant (FORT23/00034) awarded by the Programme FORTALECE funded by the Instituto de Salud Carlos III (ISCIII). Importantly, none of the funding sources disclosed has had any role in data collection, analysis or interpretation, study design, patient recruitment, writing of the manuscript, decision to submit it for publication or any other aspect pertinent to the study. None of the authors has been paid to write this article by any pharmaceutical company.

Competing interests

C.T. is currently being funded by a Miguel Servet contract, awarded by the Instituto de Salud Carlos III (ISCIII), Ministerio de Ciencia e Innovación de España (CP23/00117). She has also received a 2020 Junior Leader La Caixa Fellowship (fellowship code: LCF/BQ/PI20/11760008), awarded by ‘la Caixa’ Foundation (ID100010434), a 2021 Merck’s Award for the Investigation in MS, awarded by Fundación Merck Salud (Spain), Research Grants (PI21/01860 and PI24/01277) awarded by the Instituto de Salud Carlos III (ISCIII), Ministerio de Ciencia e Innovación de España, and co-funded by the European Union; and a FORTALECE research grant (FORT23/00034) awarded also by the ISCIII, Ministerio de Ciencia e Innovación de España. In 2015, she received an ECTRIMS Post-doctoral Research Fellowship and has received funding from the UK MS Society. She is a member of the Editorial Board of Neurology Journal and Multiple Sclerosis Journal. She has also received honoraria from Roche, Novartis, Merck, Sanofi, Immunic Therapeutics and Bristol Myers Squibb. She is a steering committee member of the ORATORIO HAND (O’HAND) trial and of the Consensus group on Follow-on DMTs. S.S.C. has received travel support to attend scientific conferences from Merck, Sanofi and Roche. S.O.R. has received speaking and consulting honoraria from Genzyme, Biogen-Idec, Novartis, Roche, Excemed and MSD; as well as research support from Novartis. R.C. is currently funded by a grant, awarded by the Instituto de Salud Carlos III (ISCIII), Ministerio de Ciencia e Innovación de España (FI/2400299). In 2023 he was awarded with a Research Training Programme from the European Charcot Foundation. In 2023 he received a grant by ‘Vall d´Hebron Institut de Recerca’. In 2021, he received an ECTRIMS Fellowship training performed during 2021–2022. He has also received speaking honoraria from Roche, Novartis, Biogen, Merck and Sanofi. P.C.M. is currently working at Bayer pharmaceuticals, but during the carrying out of the analyses reported in this paper he had nothing to disclose. A.C.C. has received a grant from Instituto de Salud Carlos III, Spain; JR19/00007. I.R. served on scientific advisory boards, received conference travel support and/or speaker honoraria from Roche, Novartis, Merck and Biogen. G.A. has received compensation for consulting services, speaking honoraria or participation in advisory boards from Roche, Bristol Myers Squibb, and UCB; and travel support for scientific meetings from Novartis, Roche, ECTRIMS and EAN. She has received speaking honoraria and consulting services or participation in advisory boards from Sanofi, Merck, Roche and Horizon Therapeutics; travel expenses for scientific meetings from Novartis, Roche, and ECTRIMS. M.C. has received compensation for consulting services and speaking honoraria from Bayer Schering Pharma, Merck Serono, Biogen-Idec, Teva Pharmaceuticals, Sanofi-Aventis, Genzyme and Novartis. A.P. has received travel funding from Roche and speaking honoraria from Novartis. He completed the ECTRIMS Clinical Training Fellowship Program in 2021 and the MSIF–ARSEP and European Charcot Foundation Fellowship Programs between 2023 and 2025. D.P. holds a research grant from the Instituto Carlos III (PI22/01709), co-funded by the European Union. J.R. has received speaking honoraria and personal compensation for participating on Advisory Boards from Biogen-Idec, Genzyme, Janssen, Merck-Serono, Novartis, Teva, Roche and Sanofi-Aventis. B.R.A. has received speaking honoraria from Merck and honoraria for consulting services from Novartis. Á.V.J. has received support for contracts Juan Rodés (JR16/00024) and receives research support from Fondo de Investigación en Salud (PI17/02162, PI22/01589 and PI25/00730) from Instituto de Salud Carlos III, Spain; and has engaged in consulting and/or participated as speaker in events organized by Novartis, Roche, Merck, Sanofi, Lundbeck, Amgen and Neuraxpharm. A.Z. has received a Rio Hortega grant, from the Instituto de Salud Carlos III, Spain (CM22/00237) and received travel expenses for scientific meetings from Biogen-Idec, Merck Serono and Novartis; speaking honoraria from Eisai; and a study grant from Novartis. J.S.G. receives research support from Instituto Salud Carlos III cofunded by the European Union (PI19/0950, PI22/0750, PI25/01664). J.S.G. has participated in the last 36 months in events funded by Merck, Roche, BMS, Sanofi, Novartis, and Cobel-Darou. J.S.G. is a member of the executive board of ECTRIMS and Editor-in-Chief of Multiple Sclerosis Journal. À.R. serves on scientific advisory boards for Novartis, Sanofi-Genzyme, Synthetic MR, Roche, Biogen and OLEA Medical; has received speaker honoraria from Bayer, SanofiGenzyme, Merck-Serono, Teva Pharmaceutical Industries Ltd, Novartis, Roche and Biogen; and is CMO and co-founder of TensorMedical. T.K. served on scientific advisory boards or as a consultant for MS International Federation and the World Health Organization, Therapeutic Goods Administration, BMS, Roche, Janssen, Genzyme, Novartis, Merck and Biogen, received conference travel support and/or speaker honoraria from WebMD Global, Merck, Sandoz, Novartis, Biogen, Roche, Eisai, Genzyme, Teva and BioCSL and received research or educational event support from Biogen, Novartis, Genzyme, Roche, Celgene and Merck. X.M. has received speaking honoraria and travel expenses for participation in scientific meetings, has been a steering committee member of clinical trials or participated in advisory boards of clinical trials in the past years with Abbvie, Actelion, Alexion, Biogen, Bristol-Myers Squibb/Celgene, EMD Serono, Genzyme, Hoffmann-La Roche, Immunic, Janssen Pharmaceuticals, Medday, Merck, Mylan, Nervgen, Novartis, Sandoz, Sanofi-Genzyme, Teva Pharmaceutical, TG Therapeutics, Excemed, MSIF and NMSS. M.T. has received compensation for consulting services, speaking honoraria and research support from Almirall, Bayer Schering Pharma, Biogen-Idec, Genzyme, Immunic Therapeutics, Janssen, Merck-Serono, Novartis, Roche, Sanofi-Aventis, Viela Bioand Teva Pharmaceuticals, Data Safety Monitoring Board for Parexel and UCB Biopharma, Relapse Adjudication Committee for IMCYSE SA. All other authors report no competing interests.

Supplementary material

Supplementary material is available at Brain online.