Inclusion of Optic Neuritis in Dissemination in Space Improves the Performance of McDonald 2017 Criteria in Hispanic People with Suspected Multiple Sclerosis

Abstract

Background:

Hispanic people compared to White people with MS are two-times more likely to present with optic neuritis (ON). ON in dissemination in space (DIS) after a single attack is not part of the current McDonald 2017 criteria.

Objective:

To evaluate if adding ON in DIS (ON-modified criteria) improves the performance of the McDonald 2017 criteria in the diagnosis of MS after a single attack of ON.

Methods:

Retrospective study of 102 patients of Hispanic background. Cases were reviewed between 2017 and 2021. Clinical ON was reported for 35 cases. ON in DIS was verified for 28 patients via MRI, optical coherence tomography, and/or visual evoked potential. We investigated the performance of the McDonald 2017 criteria and ON-modified criteria and calculated sensitivity, specificity, positive and negative predictive values, and accuracy.

Results:

The ON-modified criteria significantly improved the performance of the McDonald 2017 criteria (p=0.003) and identified an additional 9 patients. Both sensitivity and accuracy increased from 64% to 74% and 62% to 71% respectively, while specificity remained unchanged (40% [95%CI:0.10,0.70]).

Conclusion:

This study provides evidence that the inclusion of ON in DIS improved the overall performance of the McDonald 2017 criteria among Hispanic people.

Background:

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that relies on specific criteria for the accuracy of diagnosis following specified demyelinating events.(1) The original development of the McDonald criteria and subsequent revisions have been largely based on data derived from adult White European and North American populations with presentations typical of clinically isolated syndromes (CIS).(2) Validation of the criteria among populations presently underrepresented in the research literature was recognized as a high priority and unmet need in the McDonald 2017 revision.(1) Studies investigating the performance of the McDonald 2017 criteria in cohorts with Hispanic background are few and have primarily been conducted in Latin America, with a relative paucity of data from patients in the United States (US).(3-5)

Hispanic people in the US have been underrepresented in research regarding MS (6) including clinical trials.(6, 7) Hispanic people with MS in the Southwest and Western regions of the US are two times more likely to present with optic neuritis (ON) compared to non-Hispanic White people.(8-10) The exclusion of ON in dissemination in space (DIS) may therefore affect the performance of the criteria among Hispanic patients. Optimization of the criteria among this population is of particular urgency given documented delays in diagnosis relative to White patients.(10)

The occurrence of ON is associated with a high risk of conversion to MS, particularly if brain MRI demonstrates the presence of one or more asymptomatic white matter lesions.(11) Inclusion of ON as evidence of DIS has been reported to improve the sensitivity and accuracy of MS diagnostic criteria.(12-14) Importantly, improving the sensitivity of diagnostic criteria facilitates earlier initiation of treatment, which in turn improves disease outcome. Hence, the goal of the present analysis was to utilize the Alliance for Research in Hispanic MS (ARHMS) registry to determine the performance of the McDonald 2017 criteria in Hispanic patients, and to examine if including ON in DIS improves its performance.

Methods:

This was an observational, retrospective study of 102 Hispanic and Latinx patients (Hispanic throughout will be used) evaluated for a possible diagnosis of MS as part of the ARHMS at the University of Southern California (USC).(15) ARHMS is a national registry dedicated to the study of MS in people of Hispanic background and a repository for demographic, clinical (including expanded disability status scale: EDSS), biological, and radiological data. Cases were reviewed from the registry for potential inclusion by neurologists with expertise in MS (Robers, Soneji) and neuro-ophthalmology (Soneji) and then verified (Amezcua). There was no disagreement between raters. Cases were identified from among patients evaluated between 2017 and 2021. All patients included in the study had i) a clinical assessment by a neurologist at least one year after the onset of CIS and ii) at least 36 months of clinical follow-up in our center. Additional inclusion criteria were: 1) self-identified as Hispanic or Latinx, 2) had a clinically isolated syndrome (CIS) suggestive of MS such as ON, supratentorial syndrome, brainstem or cerebellar syndrome, or partial myelitis, 2) 18 years or older, and 3) had a baseline MRI within 3-6 months from onset of CIS. Patients diagnosed with neuromyelitis optic spectrum disorder (NMOSD)(17) or myelin oligodendrocyte glycoprotein antibody disorder (MOG-AD)(18) were excluded, as were patients with optic nerve lesions displaying characteristics typical of these conditions (i.e., concomitant bilateral ON, longitudinal optic nerve lesions, optic chiasm involvement). This study was approved by the institutional review board at USC.

Each MRI from which data was collected was performed on either a 1.5 or 3.0 Tesla scanner. For patients presenting with spinal cord syndrome, a specific MRI of the relevant region of the spinal cord (cervical or thoracic) was required. Lesions were identified independently by raters blinded to the clinical presentation (Amezcua, Soneji, Manouvakhova). The McDonald 2017 criteria permit the use of cortical lesions to establish DIS, but these lesions were not included in our study. MRI scans were obtained during routine care, and therefore there was significant heterogeneity in scanner strength (1.5 vs. 3.0 Tesla) and sequences obtained. Images were therefore not sufficient to reliably detect cortical lesions.

We retrospectively applied the McDonald 2017 DIS criteria requiring one or more T2-hyperintense lesions in at least 2 anatomical regions typically affected in MS (periventricular, juxtacortical, infratentorial, and spinal cord). The McDonald 2017 MRI criteria for dissemination in time (DIT) required the simultaneous presence of gadolinium-enhancing and non-enhancing lesions on a single MRI scan, or a new T2-hyperintense lesion (with or without gadolinium enhancement) on a follow-up MRI. CSF assessment for the presence of oligoclonal bands (OCB) was also used to fulfill DIT when available (CSF laboratory studies were available for 53% of patients). The presence of two or more, CSF-specific OCBs was considered indicative of intrathecal immunoglobulin synthesis.

ON-modified criteria:

We evaluated the performance of an alternate set of diagnostic criteria, namely the McDonald 2017 criteria with an allowance for ON to satisfy DIS. Medical records of study patients who had presented with clinical ON were reviewed for evidence of ON lesions. We required a history of neurological or neuroophthalmological clinical examination and supportive paraclinical evidence consisting at a minimum of an MRI demonstrating an optic nerve lesion, abnormal optical coherence tomography (OCT) suggestive of optic neuropathy, and/or visual evoked potential with P100 latencies consistent with optic neuropathy based on published references values. Those that did not have confirmatory data for ON (n=7, Figure 1) but for whom DIS could be satisfied according to the McDonald 2017 criteria, were considered to have MS. All studies were conducted as routine care.

Figure 1: Flow chart for suspected MS cases.

Of the 102 patients with suspected MS, 35 (34.3%) presented with optic neuritis(ON). Of those, 28 had paraclinical data to support ON in dissemination in space (DIS) and 7 did not (but met 2017 McDonald criteria).

Statistical Analysis:

We performed sensitivity and specificity analysis to evaluate and compare the performance in the diagnosis of MS of the McDonald 2017 diagnostic criteria with that of the ON-modified criteria. All statistical analyses were performed using SAS 9.3. True positives (TP) are those who fulfilled the 2017 McDonald criteria at the time of the first event and diagnosis of MS was validated at the time of follow-up period of 36 months. False positives (FP) are those that fulfilled the 2017 McDonald criteria at the time of the first event and the diagnosis of MS was not validated at the follow-up period of 36 months. True negatives (TN) were defined as those who presented with CIS who did not meet the 2017 diagnostic criteria and did not develop MS within the follow-up period. False negatives (FN) were those who presented with CIS who did not meet the McDonald 2017 criteria at baseline and developed MS within the pre-specified follow-up period.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and corresponding 95% confidence intervals of the two sets of diagnostic criteria using ‘senspec’ option of Freq procedure in SAS 9.3. Accuracy was calculated as [(TP+ TN) / TP+TN+FP+FN)] x 100. All patients (n = 102) were included to assess the performance of the McDonald 2017 Criteria (Figure 1). The performance of the diagnostic criteria sets were compared to each other using McNemar’s Chi-square test for paired data.

Results:

The demographic and clinical profile of the patients is shown in Table 1. Most patients were female (n=73, 71.6%) and were diagnosed with MS by age 35 years (standard error (SE) 1.02 years). The age of diagnosis ranged between 19-62 years. The average lag time between symptomatic onset and diagnosis was 2.30 years (SE 0.36 years). The mean follow-up time was 3.89 years (SE 0.77 years). A total of 17 patients were started on disease-modifying treatment (DMT) during follow-up.

Table 1:

Demographic and Clinical Characteristics (N=102)

	Non-Optic Neuritis Group (67)	Optic Neuritis Group (35)*	All Patients (102)
Female, n(%)†	44 (65.7)	29 (82.9)	73 (71.6)
Age of demyelinating event, Mean(SE)	35.4 (1.38)	34.4 (1.41)	35.0 (1.02)
Abnormal base MRI n(%)	67 (100)	35 (100)	102 (100)
EDSS, Mean (SE) †	3.23 (0.25)	2.40 (0.25)	2.94 (0.19)
Diagnosis lag, years, Mean (SE)	2.01(0.43)	2.86(0.68)	2.30 (0.36)
CSF available n(%)	40 (59.7)	15 (42.8)	55 (53.9)
Oligoclonal band positive, n (%)	24 (60)	11 (73.3)	35 (63.6)
On DMT, n (%)†	14 (20.9)	3 (8.6)	17 (16.7)

SE: Standard Error, EDSS: Expanded Disability Status, CSF: Cerebral spinal fluid, DMT: disease modifying treatment

^*7 cases ON was not confirmed in DIS

^†p-value<0.05

Of the 102 patients, 35 patients presented with optic nerve involvement (ON). Compared to the non-ON group, the ON group was more likely to be female (82.9% compared to 65.7%, p-value <0.05) and have a greater disability status (Table 1). The mean EDSS was statistically significantly higher (p-value =0.02) at study entry for those who presented with ON (Mean 3.23, SE 0.25) compared to those for whom ON was not the first event (Mean 2.4, SD 0.25). The ON group was also significantly less likely to have been started on DMT (Table 1). The pattern was similar when the analysis was restricted to cases with clinically confirmed ON (n=28) (see Supplementary Table S1).

Performance of the McDonald 2017 criteria:

Of the 102 patients, 65 participants (Table 2: TP+FP) met the McDonald 2017 DIS+DIT criteria and 59 were eventually diagnosed with MS (TP). Of the 37 (FN+TN) participants who did not meet the diagnostic criteria at the initial visit, 33 eventually were diagnosed with MS at follow-up (FN). Of all the 92 cases who were eventually identified with MS, 59 were identified using the McDonald 2017 criteria with a sensitivity of 64% [95% confidence interval (95% CI): 0.54,0.74], while of the 10 participants who never developed MS (TN+FN), 4 participants (TN) were identified as not having MS using the McDonald 2017 criteria within the data specificity of 40% (95% CI: 0.10, 0.70).

Table 2:

Performance of 2017 McDonald criteria and the ON-modified criteria in a cohort of Hispanic patients (n=102)

	TP (n)	FP (n)	FN (n)	TN (n)	Sensitivity (95%CI)	Specificity (95%CI)	PPV (95%CI)	NPV (95%CI)	Accuracy
McDonald 2017	59	6	33	4	0.64 (0.54,0.74)	0.40 (0.10,0.70)	0.91 (0.84,0.98)	0.11 (0.01,0.21)	62% (0.53-0.71)
ON-Modified†	68	6	24	4	0.74 (0.65,0.83)	0.40 (0.10,0.70)	0.92 (0.86,0.98)	0.14 (0.01,0.27)	71% (0.62-0.80)

^*TP= True Positive, FP= False Positive, FN= False Negative, TN= True Negative, PPV= Positive Predictive Value, & NPV= Negative Predictive Value.

^†ON-Modified criteria statistically significantly differed from McDonald’s 2017 criteria (p-value =0.003).

Integrating ON to DIS:

The analysis comparing the performances of the McDonald 2017 Criteria and ON-modified criteria is summarized in Table 2. Of the 33 cases for whom DIS could not be established within the McDonald 2017 criteria, 9 presented with clinical ON and had objective clinical or paraclinical evidence of optic nerve involvement. These 9 patients therefore satisfied the ON-modified criteria for a diagnosis of MS. Therefore, an additional 9 patients were identified as having DIS when the ON-modified criteria were applied. The sensitivity of the ON-modified criteria (0.74, 95% CI: 0.65, 0.83) for the detection of MS was significantly higher (p=0.003) than that of the unmodified McDonald 2017 criteria (0.64, 95% CI: 0.54, 0.74). Specificity did not differ between the two criteria sets. The accuracy of the ON-modified criteria (62%) was likewise superior to that of the McDonald 2017 criteria. The PPV and NPP associated with the ON-modified criteria were both slightly higher than the McDonald 2017 criteria.

Sensitivity Analysis:

We performed an additional restricted sensitivity and specificity analysis after removing the seven cases with clinically unconfirmed ON (N=95) (Table 3). The restricted analysis again demonstrated higher sensitivity and similar specificity of the ON-modified criteria in comparison to the McDonald 2017 criteria (p=0.005).

Table 3:

Performance of 2017 McDonald criteria and the ON-modified criteria in a cohort of Hispanic patients without disease-modifying treatment (N=85)

	TP (n)	FP (n)	TN (n)	FN (n)	Sensitivity (95%CI)	Specificity (95%CI)	PPV (95%CI)	NPV (95%CI)	Accuracy
McDonald 2017	46	4	31	4	0.60 (0.49,0.71)	0.50 (0.15,0.85)	0.92 (0.84,0.99)	0.11 (0.00,0.22)	58.8% (48.3-68.2%)
ON-Modified‡	54	4	23	4	0.70 (0.60,0.80)	0.50 (0.15,0.85)	0.95 (0.87,0.99)	0.15 (0.01,0.28)	69.3% (58.3-78.1%

^*TP= True Positive, FP= False Positive, FN= False Negative, TN= True Negative, PPV= Positive Predictive Value, & NPV= Negative Predictive Value.

^‡ON-Modified criteria statistically significantly differed from McDonald’s 2017 criteria (p-value =0.005).

Treatment with a disease-modifying therapy (DMT) can delay progression from a single demyelinating attack to multiple sclerosis and therefore could be a confounding influence on the progression of patients to disease consistent with McDonald 2017 criteria. To determine whether this was the case, we performed an additional restricted analysis excluding patients who received DMT (Table 4). The sensitivity of the ON-modified criteria (70%, 95% CI 0.60, 0.80) remained superior to that of the McDonald 2017 criteria (60%, 95% CI 0.49, 0.71) (p=0.003). Again, specificity was unchanged.

Table 4:

Performance of 2017 McDonald criteria and the ON-modified criteria among Hispanic cases after removing those with clinically unconfirmed ON (N=95)

	TP (n)	FP (n)	FN (n)	TN (n)	Sensitivity (95%CI)	Specificity (95%CI)	PPV (95%CI)	NPV (95%CI)	Accuracy
McDonald 2017	58	4	30	3	0.66 (0.56,0.76)	0.43 (0.06,0.80)	0.93 (0.87,0.99)	0.09 (0.01,0.19)	64% (54-74%)
ON-Modified†	67	4	21	3	0.76 (0.67,0.85)	0.43 (0.06,0.80)	0.94 (0.89,0.99)	0.12 (0.01,0.26)	74% (65-83%)

^*TP= True Positive, FP= False Positive, FN= False Negative, TN= True Negative, PPV= Positive Predictive Value, & NPV= Negative Predictive Value.

^†ON-Modified criteria statistically significantly differed from McDonald’s 2017 criteria (p-value =0.003).

Discussion:

This study demonstrates that the modified McDonald 2017 criteria by incorporating ON as evidence of DIS (ON-modified criteria) perform well in a Hispanic cohort. Sensitivity and accuracy were both improved, suggesting that the incorporation of ON could facilitate earlier diagnosis and treatment of MS among Hispanic people.

Several studies have investigated the performance of the McDonald 2017 criteria (1, 16-18), including several among cohorts of Hispanic patients in Latin America. (3-5) The performance of the McDonald 2017 criteria in an Argentinian cohort reported a sensitivity of 84% and a specificity of 14%.(19) It is noteworthy that our findings include a lower sensitivity (64%) but markedly higher specificity (40%). It is unclear whether our findings may be explained by regional differences in routine care and follow-up, differences in the frequency of DMT use (17% versus 95% in the Argentinian cohort), and/or the presence of differing patterns of genetic risk among Hispanic populations with differing ancestry. The improved specificity in the analysis restricted to patients not exposed to DMT supports the reliability of our results.

MS is a complex disease influenced by both genetic and environmental factors. Different patterns of risk and clinical expression have been associated with differing ancestry among patients with MS (23, 24) and ON (20). For example, genetic markers of native ancestry are significantly associated with ON as the presenting clinical event in a large Hispanic cohort.(20) Patients in the Argentinian cohort had a higher proportion of genetic markers of European ancestral background (100% European ancestry was reported for 72% of the people included).(19) While we did not account for genetic admixture, the proportion presenting with ON (27.4%) is similar to previous reports of ON frequency among Hispanic patients (18.2%-38%).(20) The Hispanic population in the US displays considerably more genetic admixture compared to other populations reflected in available data. Among Hispanic patients with MS in the US, genetic ancestry has been reported at 19% (SD 18%) Native, 72% (SD 20%) European, and 9% (SD 11%) African ancestry(20). The influence of genetic admixture and/or specific risk alleles on the performance of diagnostic criteria remains unclear.

The inclusion of ON as evidence sufficient to establish DIS has been reported to improve the overall performance of the McDonald criteria.(12-14) The Magnetic Resonance Imaging in Multiple Sclerosis (MAGNIMS) group found that the inclusion of optic nerve evaluation resulted in similar sensitivity (0.92 95% CI 0.87-0.96), and slightly lower specificity (0.26 95% CI 0.18-0.34) to the McDonald 2010 criteria.(13) In a CIS prospected study that included optic nerve involvement, both symptomatic and asymptomatic, the addition of symptomatic ON as evidence of DIS in as well as dissemination in time found the modified criteria to be more sensitive (83% vs. 74%) and accurate (81% vs 75%) with similar specificity (77%).(12) The large Barcelona cohort has also reported the added value of incorporating optic nerve in DIS. They found a slight increase in sensitivity (2017 79.2% versus modified 82.3%) and accuracy (2017 75.5% versus modified 78.1%) without a change in specificity (2017 52.4% versus modified 52.4%).(14) We were able to identify 9 (10%) additional patients with MS with our ON-modified criteria, similarly, improving sensitivity (from 64% to 74%) without a change in specificity (40% vs. 40%). Restricting our analysis by DMT exposure resulted in minor changes in sensitivity (74% to 70%) and specificity (40% to 50%), but results retained statistical significance and the pattern of improved performance was consistent. Our analysis suggests that the inclusion of ON in the determination of DIS would facilitate the goal pursued in revisions of diagnostic criteria, namely earlier diagnosis, and treatment. This may be especially important among Hispanic patients given the frequency with which ON is observed as a presenting feature.(9, 10).

Limitations

This study has several limitations. Data collection was subject to the biases inherent in retrospective review. Our cohort is primarily derived from one of the counties with the largest proportion of Hispanic and Latinx residents. 49% of Los Angeles County residents identify as Hispanic, and 75% of Hispanic residents report Mexican heritage, which is the most common Hispanic heritage in the US. (25) The tremendous heterogeneity of ancestry, cultural practice, and other characteristics among the many groups referred to as Hispanic or Latinx limits the generalizability of our findings. CSF studies were only available for 53% of patients. Hence, the performance of the McDonald 2017 criteria could have been improved if more patients had undergone lumbar puncture given that fewer false negatives would have occurred. Aversion to lumbar puncture procedures has been noted to differ by race and education, where higher education and people of White background are more likely to undergo the procedure.(21) Sociocultural factors may therefore have influenced the frequency of lumbar puncture resulting in an underestimation of the performance of both McDonald 2017 and ON-modified criteria. The acquisition of paraclinical evidence of MS among Hispanic patients may be impeded by barriers to healthcare access documented among Hispanic persons in comparison to White persons.(27) This is an important consideration in evaluating the validity of diagnostic criteria among diverse populations. All patients in our cohort were of Hispanic background and therefore may have experienced similar barriers, but experiences with said barriers were not directly studied and therefore it cannot be definitively determined whether their occurrence influenced criteria performance. Another limitation is the inclusion of patients without an MRI of the spinal cord. While spinal cord MRI is not required by current disease criteria, additional patients may have qualified for a diagnosis of MS at the time of baseline evaluation if cord lesions had been found. The diagnosis of ON can be reliably supported by MRI, OCT, and VEP despite sensitivity differences between the various paraclinical tools (28, 29). Nevertheless, in addition to the careful clinical assessment of ON, every possible effort was made to verify the presence of optic neuropathy using these common tools to avoid diagnostic errors.

In summary, our findings underscore that the McDonald 2017 criteria perform well, are predictive of MS, and can be reliably applied in the evaluation of Hispanic patients in the US. Additionally, the inclusion of ON in the determination of DIS improves sensitivity and accuracy while preserving specificity. This may be particularly relevant to Hispanic patients. These and other considerations particular to diverse populations should be carefully considered in future revisions of MS diagnostic criteria.