Remote administration of the symbol digit modalities test to individuals with multiple sclerosis is reliable: A short report

Abstract

Background

The Symbol Digit Modalities Test (SDMT) is the gold standard for cognitive screening in multiple sclerosis (MS).

Objective

Due to the recent COVID-19 pandemic and the increased need for virtual clinical visits, we examined the reliability of remote administration of the SDMT vs. standard in-person administration to individuals with MS.

Methods

Pearson’s correlation analysis was performed between SDMT scores on the in-person and remote administrations.

Results

For n = 132 participants, remote and in-person SDMT scores were strongly correlated (r = .80, p = .000).

Conclusion

Remote administration of the SDMT is a reliable cognitive screening approach in MS.

Introduction

During the pandemic, patients have been unable or reluctant to attend non-emergent outpatient appointments, requiring a rapid shift to the utilization of teleneurology.¹ The Symbol Digit Modalities Test (SDMT²), a brief measure of information processing speed, is considered the gold standard in screening for cognitive involvement in MS.³ Due to its short duration (90 seconds) and ease of administration, together with high sensitivity in detecting subtle changes in cognitive functioning in MS,^4–6 routine administration of the SDMT has been adopted by many clinicians as part of the neurological exam and routine evaluation of disease status.

We tested the reliability of remote SDMT administration to individuals with MS, comparing performances on remote and in-person administrations, delivered in the context of a large clinical trial of cognitive remediation.⁷

Materials and methods

Participants

All participants met eligibility criteria for the larger trial,⁷ including having a confirmed diagnosis of MS and at least one month outside of clinical relapse and/or steroid use. All participants provided written informed consent approved by the Institutional Review Board at Stony Brook Medicine, Stony Brook, New York.

SDMT administration

Participants were first prescreened with a remote administration of the SDMT: 1) participants were emailed a URL that directed them to our secured research website, using login information to unlock an embedded image of the SDMT that was scaled to the dimensions of their browser, and 2) participants were guided by the study technician over the phone and their responses were recorded following standard SDMT administration. Participants then attended an in-person baseline study visit, where an alternate SDMT form⁸ was administered according to standard procedures.

Results

Data for n = 132 participants (age = 50.17, range 18 to 69 years; median EDSS = 3.5, range 0.0 to 8.0) were available for analyses. See Table 1 for sample characteristics. On average, the standard in-person SDMT was administered 14 days after the remote virtual administration (mean = 14.02, SD = 10.72).

Table 1.

Demographic and clinical features of the sample.

Sample characteristics (n = 132)
Age in years (mean ± SD/range)	50.17 ± 12.26 (18–69)
Gender (% female)	78%
Education in years (mean ± SD/range)	14.97 ± 2.46 (11–20)
EDSS (median, range)	3.5 (0–8)

Pearson’s correlation analysis indicated that the remote and in-person SDMT administrations were strongly correlated (raw scores, r = .80, p < .001).

The repeated in-person SDMT administration (range = 14–75, mean = 48.13, SD = 10.73) resulted in significantly increased scores compared to the remote administration (range = 15–68, mean = 40.68, SD = 10.48), t(131) = 12.77, p < 0.001.

Discussion

This is the first study to our knowledge to compare performance on remote and in-person administrations of the SDMT. We found that the remote administration was both feasible and highly reliable, supporting administration of the SDMT during virtual visits as a general cognitive screen in MS.

There are several possible considerations for the observation of improved scores at the second in-person administration. MS commonly includes visual involvement, and the presentation of the stimuli on the computer screen may have added visual interference to result in slower responses. Alternatively, this finding may reflect an overall regression to the mean and/or practice effects, as some studies have shown that even with alternative forms, performance can improve with repeat SDMT administrations.⁹ The significant difference in scores between administrations emphasizes the need for extensive validation studies of remote test administration.¹⁰ Until such data are available, it is important to interpret results from remote administration with caution, particularly if scores are low and indicate impairment, or in the case of a significant decline from a previous in-person administration. In such instances, further confirmation should be obtained with an in-person standard administration. However, if scores are consistent through remote administration, our findings would suggest that a decline is unlikely. Currently, remote administration should not substitute in-person standardized administration, rather, it should be viewed as a highly useful proxy when telehealth is necessary, and primarily to screen for possible cognitive change.

Our approach to the remote administration, with guidance only through telephone, imposed several limitations. First, it did not allow for direct observation of test completion, preventing assurance that the participant directly and accurately participated, without exposure to stimuli for practice before the test began (e.g., during instruction). Additionally, there may have been reduced motivation for optimal performance on the task due to the lack of face-to-face contact with the examiner. However, these concerns are now largely resolved with the current standard of live videoconference for teleneurology visits.

Concerns regarding the use of telehealth platforms for cognitive testing more generally include the limited control over the testing environment (e.g., noise, distraction), privacy and confidentiality (e.g., other family members in the room), technological competency and comfort (particularly for older participants), poor internet connection, and test security. Therefore, remote testing options should be weighed carefully against these limitations on individual-case basis, and implemented only when the benefits of remote testing outweigh reliability and validity concerns.

As the SDMT was first administered remotely to all participants as part of a larger clinical trial, we are unable to separate the factors that may have affected the results of both administrations had we alternated between the order of virtual vs. in person administrations. For instance, as noted, our findings do not allow us to determine whether improved results on the second administration reflects increased difficulty inherent to remote testing (administered first), or simply, regression to the mean and/or practice effects on second administration. Additionally, with remote administration, it is important to verify the size and quality of the stimuli. In this study, we used a designated website for stimuli presentation (with more optimized web-based formats currently available), and participants viewed the testing materials on a computer screen. As many patients connect to their virtual health visits on their phone or tablet, a thorough consideration should be given to stimuli presentation.

Further extensive validation studies of remote administration procedures are needed in terms of visual interference from the screen, standardization of stimuli for presentation, and to derive appropriate normative data.¹⁰ Nonetheless, the feasibility and reliability of remote administration of the SDMT support further study of this form of remote cognitive screening in MS, which can have important and long-term clinical implications for individuals with MS with limited accessibility to in-person visits.

Conflict of Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.