Real-world validation of the 2017 McDonald criteria for pediatric MS

Abstract

Objective

To compare the diagnostic accuracy of the McDonald 2017 vs the McDonald 2010 criteria to predict a second attack of MS (clinically definite MS [CDMS]) at the first attack of acquired demyelinating syndromes (ADS).

Methods

One hundred sixty-four children (aged <18 years) with an incident attack of ADS were included in a prospective multicenter study between June 2006 and December 2016. Brain (and spinal if available) MRI was performed ≤3 months after symptom onset. Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were compared at baseline between the 2010 and 2017 criteria.

Results

Among the 164 patients, 110 patients (67%) presented without encephalopathy (ADS–, female 63%; median age 14.8 years, IQR 11.3–16.1years) and 54 (33%) with encephalopathy (acute disseminated encephalomyelitis [ADEM], female 52%; median age 4.0 years, IQR 2.6–6.1 years). Of the 110 ADS– patients, 52 (47%) were diagnosed with CDMS within a median follow-up of 4.5 years (IQR 2.6–6.7 years). The sensitivity was higher for the 2017 criteria than for the 2010 criteria (83%; 95% CI 67–92, vs 49%; 95% CI 33–65; p < 0.001), but the specificity was lower (73%; 95% CI 59–84 vs 87%; 95% CI 74–94, p = 0.02). At baseline, 48 patients fulfilled the 2017 criteria compared with 27 patients when using the 2010 criteria. The results for children aged <12 years without encephalopathy were similar. In patients with ADEM, 8% fulfilled the 2010 criteria and 10% the 2017 criteria at baseline but no patient fulfilled the criteria for CDMS.

Conclusions

The McDonald 2017 criteria are more sensitive than the McDonald 2010 criteria for predicting CDMS at baseline. These criteria can also be applied in children aged <12 years without encephalopathy but not in children with ADEM.

Classification of evidence

This study provides Class II evidence that in children with ADS, the 2017 McDonald criteria are more sensitive but less specific than the 2010 McDonald criteria for predicting CDMS.

MS is a chronic demyelinating disorder of the CNS.¹ Up to 10% of all patients with MS have their first symptoms before age 18 years.^2,3 Recently, the international panel on diagnosis of MS proposed the McDonald 2017 criteria by reviewing and revising the previous 2010 McDonald criteria.^4,5 These revised criteria include modifications to facilitate earlier MS diagnosis while attempting to preserve the diagnostic accuracy of the criteria.⁵ Important modifications included reintroducing CSF oligoclonal bands (OCBs) into the criteria as a substitute for dissemination in time (DIT) and allowing symptomatic lesions to contribute to dissemination in space (DIS) and DIT. Furthermore, cortical lesions have been combined with the juxtacortical lesion component to demonstrate DIS.

The applicability of the adult McDonald 2010 criteria in children was supported by several studies.^6–12 These criteria were implemented in the revised 2012 diagnostic criteria for children with acquired demyelinating syndromes (ADS) proposed by the International Pediatric MS Study Group (IPMSSG).¹³ ADS encompass the first attack of demyelination in the CNS.^13,14 According to the IPMSSG criteria, MS diagnosis based on the first baseline MRI in children with a first attack should be performed with caution for children aged <12 years and patients presenting with acute disseminated encephalomyelitis (ADEM) because of a lower specificity and sensitivity of the McDonald 2010 criteria in these groups.¹³

A Canadian study reported that the revised McDonald 2017 criteria apply well in children with a first attack of ADS across the age span.¹⁵ Validation of these criteria in different study populations is imperative because of the long-term administration of disease-modifying treatment (DMT) after MS diagnosis. Overtreatment in patients must be prevented, as well as undertreatment, because of delayed diagnosis. Early and accurate identification of patients with MS is therefore essential.

We aimed at evaluating the diagnostic accuracy of the revised 2017 MS criteria in children with ADS at the time of the first presentation. Extra attention is paid to children younger than age of 12 years and to patients presenting with ADEM.

Methods

Patients and design

Children younger than 18 years with a first attack of demyelination were consecutively included in the nationwide, multicenter, and prospective study for children with ADS between June 2006 and December 2016 (PROUD-kids study).^16,17 Patients were assessed at baseline and were prospectively followed up. MRI was performed within 3 months after symptom onset as part of a routine diagnostic process to rule out alternative diagnoses. Patients with alternative nondemyelinating disorders were excluded from our study. All patients had a follow-up (FU) duration of at least 1 year because the interval between the first and second attacks in pediatric MS is typically less than 1 year.^18–21

Patients were included for analysis when presenting with ADS without (ADS–) and with encephalopathy (ADEM).¹³ Patients with neuromyelitis optica spectrum disorders or with relapsing disease other than MS were excluded from analyses (e.g., patients presenting with relapsing anti–MOG antibody related disorders) because there is emerging evidence that these patients have a distinct clinical phenotype.^22,23

Standard protocol approvals and patient consents

The PROUD-kids study protocol was approved by the Ethics Committee Erasmus MC, Rotterdam, and by the other participating centers in the Netherlands. Written informed consent was obtained from all patients and/or their families.

Definitions

ADS in children encompass the first attack of demyelination in the CNS, including patients presenting with (ADS+) and without encephalopathy (ADS–). ADEM was defined as a polyfocal onset with encephalopathy (ADS+).¹³ Clinically definite MS (CDMS) was defined as a second attack of MS, with 2 nonencephalopathic confirmed attacks with clinical evidence of 2 separate lesions.¹³ ADS– patients who remained monophasic were defined as monophasic ADS–.¹³ Patients were reassessed annually. The patients were instructed to contact the outpatient clinic if new symptoms occurred to be clinically assessed. A relapse was defined as new neurologic deficits or subacute worsening of existing symptoms after 30 days of improvement or stable disease, without evidence of an alternative diagnosis.²⁴

Procedures

Brain and spine MRI scans were performed at 1.5 Tesla scanners. Available T1-, axial T2-, axial and/or sagittal fluid-attenuated inversion recovery (FLAIR)-, and T1-weighted images with gadolinium administration were evaluated centrally. The MRI scan closest to the date of symptom onset was evaluated as the baseline scan.

For DIS, all baseline MRI scans were scored using the McDonald 2010 criteria and the modified components as described in the revised McDonald 2017 criteria for the first MRI scan (table 1). MRI techniques, such as double inversion recovery, that are required to reliably demonstrate cortical lesions were not part of the routine MRI protocol. Therefore, the cortical lesion component was not taken into account in our analyses. If a spinal MRI was performed within 30 days after or before brain MRI, this scan was taken into account in scoring the DIS components. For DIT, all MRI scans with post-gadolinium T1 images were used; or scans that did not have gadolinium administered, but did not show any FLAIR/T2 hyperintense lesions either. CSF analyses for OCBs were performed in local laboratories using isoelectric focusing.²⁵ OCB status was considered positive if there were ≥2 unique bands in CSF compared with serum.

Table 1.

Baseline MRI criteria for MS diagnosis derived from the McDonald 2010 and revised 2017 criteria

Rationale

We set out to analyze our data in a manner that is representative for clinical practice. As acknowledged by the International Panel on Diagnosis of MS, spinal cord MRI and/or lumbar puncture was not performed in every case depending on clinical presentation and was left to the decision of the local treating physician.⁵ We did not exclude patients without a spinal cord MRI or lumbar puncture because this would probably introduce selection bias in our study.

DIS was based on 3 parameters (periventricular, juxtacortical, and infratentorial) or 4 including spinal localization if a spinal MRI was performed. We allowed OCB status to contribute to the fulfillment of DIT 2017 in patients with no gadolinium enhancement.

Statistical analyses

For statistical analyses, we used SPSS software, version 24.0 (SPSS Inc.) and GraphPad Prism5. CDMS diagnosis was used as an endpoint for all following analyses. For group comparisons, the χ² test and Fisher exact test were used for categorical data. The Mann-Whitney U test was used for continuous data.

Diagnostic performance and accuracy

Patients fulfilling the diagnostic criteria at the time of the first attack with a subsequent diagnosis of CDMS during FU were considered true positives. False positives did fulfill the diagnostic criteria for MS at baseline MRI but were not diagnosed with CDMS during FU. Patients who did not fulfill the diagnostic criteria at baseline MRI and who were not diagnosed with CDMS during FU were considered true negatives. False negatives were patients who did not fulfill the diagnostic criteria on baseline MRI but were diagnosed with CDMS during FU.

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were calculated, including a 95% CI.

Comparison between the sensitivity and specificity for the 2010 and 2017 criteria was made using the McNemar test. Kaplan-Meier analysis was used to analyze the time to CDMS diagnosis using the 2010 and 2017 criteria (log-rank test). Patients who did not meet the criteria for CDMS diagnosis during FU were considered censored observations. Cox hazard regression analyses were performed to calculate hazard ratios (HRs) for time to MS diagnosis based on the McDonald 2010 criteria (applied at baseline), McDonald 2017 criteria (applied at baseline), and CDMS diagnosis. p Values < 0.05 were considered statistically significant.

Data availability statement

The raw data used in this article could be shared in an anonymized format by request from a qualified investigator.

Results

Patient characteristics

The inclusion and exclusion process of eligible ADS children is displayed in figure 1. Among the 164 included patients with a first demyelinating event, 54 patients (33%) presented with encephalopathy (ADS+) and were diagnosed with ADEM. The other 110 (67%) were ADS– at the time of inclusion. Of these, 52/110 (47%) were diagnosed with CDMS during FU (median FU time, 4.5 years, interquartile range (IQR) 2.6–6.6 years). The median time to CDMS diagnosis was 10.2 months (IQR 3.8–20.7 months). None of the patients with ADEM had a second attack within a median FU of 5.1 years (IQR 2.7–7.8). Patient characteristics and statistical comparisons between monophasic ADS– and CDMS are displayed in table 2.

Figure 1. Flow chart for patient inclusion.

Table 2.

Patient characteristics

DIS and DIT performance on baseline MRI

All test characteristics of analyses with and without children aged <12 years are presented in table 3.

Table 3.

Test characteristics of the McDonald 2010 and McDonald 2017 criteria

Dissemination in space

Hundred and ten ADS– patients were included for this analysis, of whom 52 were diagnosed with CDMS during FU (47%). Spinal MRI was included when available. The 2010 DIS criteria were fulfilled in 54 patients (49%). Of these, 41 were diagnosed with CDMS (76%). Additional 16 patients fulfilled the 2017 DIS criteria (n = 70, 64%), and 48 of these 70 patients were diagnosed with CDMS (69%). The 2017 DIS criteria yield an increase in sensitivity of 13% compared with the 2010 criteria (p = 0.1) and a loss in specificity of 16% (p = 0.008).

A subanalysis evaluating the fulfillment of DIS in patients who had T1 images with gadolinium administration (n = 93) yielded no difference in the results.

Of the 110 ADS– patients, 61 patients (55%) had spinal cord imaging, and spinal lesions were detected in 45/61 scans (74%). In a subgroup analysis, we included only patients who underwent a spinal scan. The test characteristics of DIS were comparable to the results mentioned earlier.

Dissemination in time

To evaluate the DIT for both criteria, we selected ADS– patients who had T1 images with gadolinium administration (n = 93). Of these patients, 41 were diagnosed with CDMS during FU (44%). The 2010 DIT criteria were fulfilled in 35 patients (38%). Of these, 23 were diagnosed with CDMS (66%). Regarding the difference between the symptomatic and asymptomatic lesions for the DIT component, we observed that the 2017 DIT criteria (excluding OCBs) yielded 3 more patients fulfilling the DIT criteria (n = 38, 41%), of whom 26/38 were diagnosed with CDMS (68%). A major increase in patients fulfilling DIT with the 2017 criteria is caused by allowing OCBs to contribute to DIT when no gadolinium enhancement was present: an additional 20 patients fulfilled 2017 DIT criteria at baseline (n = 58 patients, 62%). Thirty-seven (64%) of these 58 patients were diagnosed with CDMS.

The test characteristics for 2010 and 2017 DIT criteria, excluding OCB status, were similar. When adding OCB status, the DIT criteria yield 27% in sensitivity (p < 0.001) but lost 17% in specificity (p = 0.004) (table 3).

McDonald 2010 vs McDonald 2017 criteria

To evaluate the McDonald 2010 and 2017 criteria, we selected ADS– patients who had T1 images with gadolinium administration (n = 93). The McDonald 2010 DIS + DIT criteria were fulfilled in 27/93 patients at baseline (29%), of whom 20 were diagnosed with CDMS (74%) after a median FU of 4.5 years (IQR 2.6–7.1 years).

The McDonald 2017 criteria identified 21 additional patients who fulfilled the criteria at baseline (n = 48, 52%) compared with the 2010 criteria, and 34/48 patients (71%) were diagnosed with CDMS. The sensitivity was higher in the McDonalds 2017 criteria (83% vs 49%; p < 0.001), and the specificity was lower (73% vs 87%; p = 0.02) (table 3).

The 7 patients who caused the loss in specificity were identified (fulfilling the 2017 criteria and not the 2010 criteria at baseline, but not having a second attack during an FU of median 2.7 years [IQR: 1.2–6.5 years]). New lesions on subsequent MRI were observed in 4 of these 7 patients during FU. The other 3 patients did not undergo a second MRI.

Of the 41 patients diagnosed with CDMS, the McDonald 2010 criteria led to the identification of 20 patients (49%) at baseline. A second attack occurred in 17/20 patients (85%) within 3 years of FU and in 19/20 (95%) within 5 years. With the McDonald 2017 criteria, 34/41 (83%) patients with CDMS were identified at baseline. At 3 and 5 years of FU, 31/34 (91%) and 33/34 (97%) had a second attack within 3 and 5 years.

Only 1 patient who fulfilled the 2010 and 2017 diagnostic criteria at baseline did not have a second attack (CDMS) within 5 years of FU, yet this patient showed new MRI lesions on FU scans.

The survival curves for CDMS diagnosis, the McDonald 2010 criteria, and the revised McDonald 2017 criteria on baseline MRI are presented in figure 2. MS diagnosis could be made earlier in patients with ADS using the 2017 than the 2010 criteria. For both criteria, the HRs for the DIS, DIT, and full criteria at baseline are displayed in table 4.

Figure 2. Time from incident ADS event to MS diagnosis.

Survival curves for time from incident ADS event to MS diagnosis according to the McDonald 2010 criteria (at baseline), McDonald 2017 criteria (at baseline), and CDMS. ADS = acquired demyelinating syndrome; CDMS = clinically definite multiple sclerosis.

Table 4.

Hazard ratios for CDMS diagnosis after applying DIS, DIT, and full criteria (DIS + DIT) for McDonald 2010 and 2017 on baseline MRI

Contribution of symptomatic enhancing lesions, OCBs, and spinal cord imaging

As described earlier, the McDonald 2017 criteria identified 21 additional patients (n = 48) compared with the 2010 criteria (n = 27).

These 21 patients were identified with the contribution of 2 major changes in the criteria. First, symptomatic lesions can be included to demonstrate DIS and DIT for the McDonald 2017 criteria. This led to 7 more MS diagnoses (n = 34) at baseline compared with the 2010 criteria (n = 27). Of these 34 patients, 25 were diagnosed with CDMS (74%).

Second, OCBs are allowed to be taken into account when assessing DIT. This led to the other 14 patients who fulfilled the McDonald 2017 criteria at baseline (total n = 48) and caused the major increase in the sensitivity of the criteria. Among these 48 patients, 34 (71%) were diagnosed with CDMS.

No additional patients fulfilled the McDonald 2010 criteria depending on spinal imaging. In contrast, the presence of a spinal cord lesion contributed to fulfilling the McDonald 2017 criteria at baseline in 6 patients. Three of these patients were eventually diagnosed with CDMS. Thus, 3/41 (7%) patients with future CDMS fulfilled the DIS component at baseline by performing spinal cord imaging.

Relevant subgroup analyses

DMT use before CDMS

DMTs have the potency to postpone a second attack and could therefore influence CDMS diagnosis. We performed a subgroup analysis after excluding patients who received DMT before CDMS diagnosis (exclusion n = 12). In the included patients (n = 81), test characteristics for the full criteria (DIS + DIT) remained comparable to the total group (including DMT use before CDMS) for both the 2010 (sensitivity 41% vs 49%; specificity 87% for both selections) and 2017 criteria (sensitivity 83% and specificity 73% for both selections).

Analysis in ADS– patients aged <12 years

The IPMSSG recommended that the McDonald 2010 criteria at baseline should be used with caution in patients aged <12 years. We applied the novel 2017 criteria to the group of patients younger than 12 years only after exclusion of ADEM cases.

Seven of 28 ADS– patients (25%) were diagnosed with CDMS. Five patients (18%) fulfilled the baseline criteria for 2010, and 4 of them (80%) fulfilled the criteria for CDMS. Nine patients fulfilled the 2017 criteria (32%), and 7 (78%) were diagnosed with CDMS.

The test characteristics were even better in children aged <12 years than in the total group for both McDonald 2010 criteria (sensitivity 57% vs 49%; specificity 95% vs 87%) and 2017 criteria (sensitivity 100% vs 83%; specificity 91% vs 73%).

Analysis in the ADEM subgroup

Only patients with ADEM are included in this analysis (n = 54). DIS was fulfilled for the 2010 criteria and 2017 criteria in 24 (44%) vs 28 (52%) patients with ADEM, respectively.

Regarding DIT, 7/49 (14%) patients with ADEM fulfilled the 2010 DIT criteria and 9/49 (18%) fulfilled the 2017 DIT criteria, including OCB status. Of note, 4/49 (8%) patients would have fulfilled the McDonald 2010 criteria and 5/49 (10%) the McDonald 2017 criteria (gain of 1 patient due to OCBs). None of the patients fulfilled the criteria for CDMS during FU.

Discussion

We investigated the application of the novel McDonald 2017 criteria for pediatric MS in clinical practice. We show that the McDonald 2017 criteria have a higher sensitivity than the previous 2010 criteria for CDMS diagnosis for pediatric patients (83% vs 49%, p < 0.001). However, the specificity was lower (73% vs 87%, p = 0.02). Overall, the diagnostic accuracy of the 2017 criteria was higher than that for the 2010 criteria (77% vs 70%). The revised criteria are easier to apply than the McDonald 2010 criteria, mainly because of the major change of accepting all lesions to contribute to DIS and DIT, without taking the clinical symptoms into account. Moreover, we show that the 2017 McDonald criteria lead to more MS diagnosis at baseline; therefore, MS diagnosis can be made earlier using the 2017 criteria.

A high sensitivity is important to start DMT as soon as possible,²⁶ which might lead to overtreatment in the group of patients who have a less active clinical disease course. Our main findings are supported by a recent extensive study in a cohort of patients with ADS that also included evaluation of the applicability of the McDonald 2017 criteria.¹⁵ We validate their finding that the sensitivity is increased and specificity is decreased mainly by including OCB status into the 2017 criteria. However, the specificity of the McDonald 2017 criteria in our study is somewhat lower. This is probably due to our study design by taking CDMS, a more clinical primary endpoint instead of new T2 lesions on a second MRI. We identified a minor subgroup of 7 patients who were responsible for the loss in specificity and who had an FU duration of 2.3 years. New lesions on subsequent MRI were observed in 4 of these patients. The other 3 patients did not undergo a second MRI. However, given the presence of typical MS lesions at baseline and OCB positivity in these 3 patients, it is quite likely that these patients would also have developed new MRI lesions after a longer FU. Taken together, we believe that initiation of DMT based on the novel criteria is warranted. Yet, clinicians should be aware that with the McDonald 2017 criteria, more patients will be identified at baseline and that a proportion of these patients will clinically have a less active disease course.

Performing spinal MRI led to fulfillment of the McDonald 2017 criteria at baseline in 3/41 (7%) patients with CDMS, but did not have additional value in the McDonald 2010 criteria, in line with another study that indicated limited value of spinal cord imaging in the McDonald 2010 criteria in children.⁶ Fadda et al.¹⁵ argued whether spinal cord acquisition would meaningfully add to the performance of the McDonald 2017 criteria because only a real small proportion of their patients fulfilled the criteria based on spinal cord MRI. However, in our cohort, 7% of the patients with CDMS could have been identified at baseline, which could significantly reduce the time to diagnosis in these patients. The exact place of spinal cord imaging as part of the MS diagnostic procedure deserves further investigation.

Our data show that approximately 10% of the patients with ADEM fulfill the McDonald 2010 and 2017 criteria at baseline. However, no patient with ADEM was diagnosed with CDMS in our cohort. The IPMSSG 2012 criteria explicitly mention not to apply the McDonald 2010 criteria to patients with ADEM. Our data support this view for the McDonald 2017 criteria to prevent the incorrect initiation of treatment in these monophasic patients.

Regarding age, ADS– patients aged <12 years seemed to have better accuracy for the McDonald 2017 criteria than the total group (sensitivity 100% vs 90%, specificity 81% vs 60%, PPV both 64%, NPV 100% vs 89%), despite the small sample size for this analysis. This implies that both the McDonald 2010 criteria and McDonald 2017 criteria can be used across the age span, including children aged <12 years with ADS, excluding patients with ADEM, in keeping with the results from previous studies.^12,15,27

Our study has several limitations. The choice of brain and/or spinal MRI, inclusion of contrast or not, and the decision to include testing for CSF OSBs were left to the discretion of the treating physician. For example, spinal imaging is not always justified (sedation may be needed), CSF is not always tested in isolated optic neuritis, and a paired serum sample is not always available with CSF (e.g., in the case of exclusion of suspected infection). Therefore, similar to other studies on the diagnostic criteria for pediatric MS, there was not complete coverage of all potentially relevant parameters. However, our main goal was to evaluate the revised 2017 criteria in real-world data; therefore, we did not exclude patients from the analyses. Instead, a few subanalyses have been performed. Despite our considerable FU duration of median 4.6 years, it is possible that some patients may develop a second attack in the future. DMT could have postponed CDMS diagnosis; however, our subanalysis excluding these patients showed no differences in test characteristics. Second, the PROUD-kids study did not have a standardized MRI FU; therefore, we did not take FU MRIs into account for the analysis of the 2010 and 2017 criteria. However, the scope of this article was to analyze the diagnostic accuracy for CDMS at the first attack of ADS.

Our data suggest that the McDonald 2017 criteria perform well in children. MS diagnosis can be made earlier and leads to a higher number of patients with MS at baseline. Both the McDonald 2010 and 2017 criteria show similar results for the patients aged <12 years presenting with ADS and can therefore be applied in this population as well. As proposed by the IPMSSG for the McDonald 2010 criteria, application of both 2010 and 2017 criteria in patients with ADEM should be avoided.

Glossary

ADEM

acute disseminated encephalomyelitis

ADS

acquired demyelinating syndrome

CDMS

clinically definite MS

DIS

dissemination in space

DIT

dissemination in time

DMT

disease modifying treatment

FLAIR

fluid-attenuated inversion recovery

FU

follow-up

HR

hazard ratio

IPMSSG

International Pediatric MS Study Group

NPV

negative predictive value

OCB

oligoclonal band

PPV

positive predictive value

Appendix 1. Author contributions

Appendix 2. Coinvestigators

Study funding

The study was supported by the Dutch MS Research Foundation. This study was not industry sponsored.

Disclosure

Y.Y.M. Wong, C.L. de Mol, R.M. van der Vuurst de Vries, E.D. van Pelt, and I.A. Ketelslegers report no disclosures. C.E. Catsman-Berrevoets received travel funding from the European Paediatric Neurology Society. R.F. Neuteboom served on the scientific advisory board of LAREB and data safety monitory board of the EXCEL study. R.Q. Hintzen participated in trials with Biogen Idec, Merck Serono, Roche, and Novartis and is on the editorial board of Multiple Sclerosis and Related Disorders. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/NN.