Abstract
Clinical trials in multiple sclerosis (MS) have been including digital technology tools to overcome limitations in treatment delivery and disease monitoring. In March 2020, we conducted a systematic search on pubmed.gov and clinicaltrials.gov databases (with no restrictions) to identify all relevant published and unpublished clinical trials, in English language, including MS patients, in which digital technology was applied. We used “multiple sclerosis” and “clinical trial” as the main search words, and “app”, “digital”, “electronic”, “internet” and “mobile” as additional search words, separately. Digital technology is part of clinical trial interventions to deliver psychotherapy and motor rehabilitation, with exergames, e-training, and robot-assisted exercises. Digital technology has been used to standardise previously existing outcome measures, with automatic acquisitions, reduced inconsistencies, and improved detection of symptoms (e.g., electronic recording of motor performance). Other clinical trials have been using digital technology for monitoring symptoms that would be otherwise difficult to detect (e.g., fatigue, balance), for measuring treatment adherence and side effects, and for self-assessment purposes. Collection of outcome measures is progressively shifting from paper-based on site, to internet-based on site, and, in the future, to internet-based at home, with the detection of clinical and treatment features that would have remained otherwise invisible. Similarly, remote interventions provide new possibilities of motor and cognitive rehabilitation.
Keywords: multiple sclerosis, clinical trial, digital technology, outcome measures
1. Introduction
Multiple sclerosis (MS) is the most common cause of neurological disability in young adults [1]. MS symptoms encompass a wide range of neurological deficits including weakness, unbalance, visual disturbances, etc. [2]. A number of clinical features (e.g., fatigue, sensory symptoms, cognitive impairment, pain, and bladder dysfunction) are difficult to evaluate on clinical examination, and, thus, are frequently named invisible symptoms, but affect daily functioning, social life, close relationships, and leisure activities [3,4]. As such, MS patients require long-term, multidisciplinary care in both clinical and community settings [5], with the use of disease modifying treatments (DMTs) aiming at reducing the risk of relapses and, at least in part, halting disability progression, along with symptomatic treatments and rehabilitation [2].
Digital technology includes both the delivery of interventions and the collection of data (e.g., outcome measures), using a variety of digital tools, including smartphones, websites, apps, and wearable devices [6,7,8]. A number of health interventions have already leveraged the use of digital technology devices, methods and systems (e.g., mobile apps, digital health, e-health, internet-based interventions, etc.) [9,10,11,12], and have targeted patients, healthcare providers, health system and resource managers, and data services [9,13]. This would be particularly important for people with MS living in geographically isolated communities, who have difficulties in accessing rehabilitation services and have limited clinical trial opportunities [14,15,16,17,18]. As such, digital technology holds promise for remote interventions both for rehabilitation purposes (physical and cognitive) and to improve the overall clinical management, focusing on aspects that would be otherwise difficult to address in clinical practice [10].
Limitations of clinical outcome measures are widely known and could be possibly overcome by digital technology. The most common clinical outcome measures in MS clinical trials are the Annualised Relapse Rate (ARR) (i.e., relapse count over time), and the Expanded Disability Status Scale (EDSS) (a clinician-rated categorical scale ranging from 0 (normal neurological exam) to 10 (death resulting from MS)). However, clinical assessments in clinical trials and practice are usually run at specific intervals (e.g., 3 to 6 months), with the risk of missing mild relapses or subtle disease progression; indeed, more frequent consultations are frequently precluded by time, cost and geographical restraints [5,19]. Besides, traditional outcome measures do not reflect insidiously subtle progression of the disease and may fail to capture transient symptomatic and performance fluctuations affecting people with MS day-to-day [20]. For instance, the EDSS is clinically relevant and accessible to neurologists, but it is burdened by low intra- and inter-rater reliability at the lower end of the scale (e.g., early phases of MS) [21], and non-linear design at the upper levels, assigning a relatively large weight to walking abilities when compared with upper limb function [22]. While DMTs have proven anti-inflammatory activity (e.g., on ARR), the effect on EDSS is negligible, suggesting these measures lack of sufficient power to detect significant treatment effect [23,24,25]. Also, the EDSS does not capture invisible symptoms of MS [26]. More recently, to overcome these limitations, several composite outcome measures were developed, including both motor and cognitive tests, aiming to provide more complete and sensitive-to-change assessments [27]. For instance, the Multiple Sclerosis Functional Composite (MSFC) includes leg function by walking a short distance (“Timed 25-Foot Walk”, T25FT), arm function using peg-board test (“9-Hole Peg Test”, 9HPT), and attention/concentration test (“Paced Auditory Serial Addition test”, PASAT). Still, composite measures are time consuming, and do not account for some MS symptoms. Not least, in the past years, clinical trials have been using patient-reported outcome measures (PROMs) to estimate common and disabling symptoms, such as fatigue, depression and pain, which are difficult to detect on clinical consultation in the absence of standardized data gathering [27,28,29,30]. Thus, more precise clinical measures are urgently needed [31].
Considering limitations of conventional interventions and clinical measures, digital technology has become increasingly used to capture the complexity of MS in clinical practice, observational studies and clinical trials [32,33]. We specifically decided to focus on the use of digital technology in clinical trials, that are a turning point for outcome measures to move to clinical practice. For instance, the lesion count on brain MRI was developed in early 90s [34], and, then, widely applied in clinical trials, before becoming clinical practice [27]. Similarly, brain atrophy and neurofilament have been initially developed on clinical trial populations, and are now progressively becoming clinical practice [23,27,35,36]. Thus, studying current advancements in digital technology in clinical trials will help the understanding of possible future changes in MS clinical practice.
In the present review, we aim to—(1) systematically review concluded and ongoing MS clinical trials using digital technology (e.g., apps, mobile devices, electronic tools, internet, etc.) as either an intervention, or an outcome measure; and (2) narratively discuss how digital technology has been changing MS clinical trial design and outcome measures, along with limitations and future perspectives.
2. Materials and Methods
In March 2020, we conducted a systematic search on two different databases (pubmed.gov for published studies and clinicaltrials.gov for unpublished/ongoing studies); no search restrictions were applied (e.g., year range, country). We set “multiple sclerosis” and “clinical trial” as the main search words, and “app”, “digital”, “electronic”, “internet” and “mobile” as additional search words, separately. We identified all relevant published and unpublished/ongoing clinical trials and observational studies registered as clinical trials (e.g., NCT number). Following search results, we included only studies in the English language, and excluded reviews and other manuscripts. All search outputs (i.e., abstract, full paper, study design) were reviewed by two independent reviewers (MDA and MM) for consistency with study objectives and relevant documents were selected.
3. Results
We included thirty-five clinical trials—fifteen phase 2 clinical trials, one phase 3 clinical trial, and one phase 4 clinical trial. We also included sixteen observational studies which were registered on clinicaltrials.gov. Included studies are reported in Table 1. We excluded four studies that did not include any digital technology tool (Figure 1).
Table 1.
Title | MS Phenotype |
Other Conditions Included | Age | Study Design | Ref. |
---|---|---|---|---|---|
MAIN INTERVENTION | |||||
Haptic feedback on robot (I-TRAVLE), arm strength and endurance training, 3 times a week for 8 weeks | All MS | Paralytic Stroke | >18 | Interventional Open Label Single Group | [37] |
Resistance-endurance training at home via internet (e-Training), 10–60 min | All MS | 18–65 | Interventional Randomized Parallel Assignment | [38] | |
Body-weight support robot (LEAP), usability study, therapist, observer and patient questionnaire | All MS | Spinal Cord Injuries Cerebral Palsy Parkinson Disease Stroke People with impaired lower extremity function |
5–80 | Interventional Open Label Single Group | [39] |
Home-based movement training (MOTOmed), 12 weeks | All MS | >18 | Interventional Randomized Parallel Assignment | [40] | |
Processing speed and attention treatment, digital Tablet-based game, 6 weeks | All MS + CIS | Cognitive Decline | 18–71 | Interventional Randomized Parallel Assignment | [41] |
Internet-based intervention to increase physical activity (eFIT), 12 weeks | All MS | >18 | Interventional Randomized Parallel Assignment | [42] | |
Web-based behavioral lifestyle intervention (POWER@MS1), 12 months | All MS | 18–65 | Interventional Randomized Parallel Assignment | [43] | |
Internet-delivered Cognitive Behavioral Intervention for Sleep Disturbance (iSLEEPms), 4 weeks | All MS | >18 | Interventional Randomized Parallel Assignment | [44] | |
Game Based Virtual Reality Exercises (USE-IT) added to rehabilitation, 4 weeks | All MS | 18–65 | Interventional Randomized Parallel Assignment Open Label | [45] | |
(MORE STAMINA) Mobile App for Fatigue Management, adherence, 60 days | All MS | Fatigue Chronic Conditions |
18–80 | Observational Prospective | [46] |
STANDARDASING PREVIOUS OUTCOME MEASURES | |||||
Fall prevention exercise and education program with electronic diary, 8 weeks | All MS | 18–89 | Interventional Open Label Single Group | [47] | |
Comparison between Mobile Multiple Sclerosis Functional Composite and classic | Relapsing-remitting MS | 18–65 | Interventional Randomized Crossover Assignment Open Label | [48] | |
Usability of the Multiple Sclerosis Performance Test Device | All MS + CIS | >18 | Observational Prospective | [49] | |
(Myo Armband) Wearable Biosensor to Track and Quantify Limb Dysfunction | All MS | 18–65 | Interventional Open Label Single Group | [50] | |
Change in the Arm Function in Multiple Sclerosis Questionnaire after 4 weeks of Home-Based Dexterity Training via Mobile App | All MS | 18–75 | Interventional Randomized Parallel Assignment | [51] | |
Automated EDSS Score Calculation Using a Smartphone Application vs paper version | Progressive MS | >18 | Observational Prospective | [52] | |
Patient Centered Outcomes Analysis using BeCare Mobile App, 6 months | All MS | 18–65 | Observational Prospective | [53] | |
Validation of the BeCare Multiple Sclerosis Assessment App | All MS | 18–75 | Observational Prospective | [54] | |
DETECTION OF INVISIBLE CLINICAL AND TREATMENT FEATURES | |||||
Realtime Digital Fatigue Score as outcome measure | All MS | 18–70 | Interventional Randomized Crossover Assignment | [55] | |
(NA-NOSE) Artificial olfactory system chemical sensor for the detection and identification of Multiple Sclerosis by Respiratory Samples | All MS | 18–60 | Observational Case-Only Cross-Sectional | [56] | |
Evaluation of adherence through BetaPlus Program elements (website) | Relapsing-remitting + Secondary Progressive MS |
>18 | Observational Prospective | [57] | |
Portuguese evaluation of adherence through BetaPlus Program elements (website) | Relapsing-remitting MS + Secondary Progressive MS |
>18 | Observational Prospective | [58] | |
Electronic measure of needle disposals (MEMS TrackCaps) as outcome measure | All MS + CIS | 18–70 | Interventional Randomized Parallel Assignment | [59] | |
Evaluation of adherence through the Use of an Electronic Diary, one year | All MS | 18–70 | Interventional Randomized Parallel Assignment Open Label | [60] | |
Fall Detection through Electronic Fall Detector, 6 months | All MS | >18 | Interventional Randomized Parallel Assignment Open Label | [61] | |
Comparing a Smartphone Application with the Composite MSFC Score, 90 days | All MS | >18 | Observational Prospective | [62] | |
Patient Reported Outcomes Measurement Information System via tablet, 6 weeks | All MS | Fibromyalgia Osteoarthritis Sjögren’s Syndrome Parkinson’s Disease |
18–76 | Interventional Randomized Parallel Assignment | [63] |
Number of Active and Passive Tests Conducted by patients on smartwatch and smartphone | All MS | 18–55 | Interventional Non-Randomized Parallel Assignment Open Label | [20] | |
Change in number of reports of adverse drug reactions through app (My eReport France), 6 months | Relapsing-remitting MS | >18 | Interventional Randomized Parallel Assignment Open Label | [64] | |
Betaferon adherence measured by mobile app (PROmyBETAapp), 6 months | Relapsing-remitting MS | >18 | Observational Prospective | [65] | |
Digital Assessment Multiple Sclerosis 3 (DAMS-3) vs MSFC | All MS | 18–60 | Interventional Non-Randomized Sequential Assignment Open Label | [66] | |
Internet collection of Patient Reported Outcomes (SymptoMScreen) as outcome measure, 192 weeks | Progressive MS | 18–65 | Interventional Randomized Single-Group Assignment Open Label | [67] | |
Ascertaining Medication Usage & Patient Reported Outcomes Via the myBETAapp, 12 weeks | Relapsing-remitting MS | >18 | Observational Prospective | [68] | |
Evaluation of the Impact of Patients’ Training by Nurse on Adverse Drug Reaction Reporting by Patient Via a Mobile Application (VIGIP-SEP2), 3 months | All MS | >18 | Interventional Randomized Parallel Assignment Open Label | [69] | |
Promotion and evaluation of adherence via Electronic Pill Bottle, 90 days | Relapsing-remitting MS | >18 | Interventional Randomized Parallel Assignement Open Label | [70] |
Looking at clinical trial inclusion criteria, studies included patients of any age (from 5 years, upwards). MS was the only disease in 31 out of 35 studies (88%), while the remaining 4 (12%) also included other neurological conditions, such as Parkinson’s disease, essential tremor, fibromyalgia, spinal cord injury, and stroke. Looking at clinical phenotypes of MS, six studies out of 35 (16%) only included relapsing-remitting MS, four studies (12%) included progressive MS, 22 studies (63%) included all phenotypes of MS, and three studies (9%) included clinically isolated syndrome (CIS) along with other MS subtypes.
From the preliminary review of selected studies, digital technology in MS clinical trials had three main purposes—testing digital technology as an intervention, standardising previously existing outcome measures, and detection of invisible clinical and treatment features (Figure 2).
3.1. Main Intervention
We found 10 ongoing studies out of 35 (29% of included studies) using digital technology as the main intervention. In particular, digital technology has been used as the main intervention for rehabilitation and psychotherapy.
A number of studies are evaluating the possibility to improve adherence and effectiveness of rehabilitation using game-based virtual reality exercises on motor function [45], fatigue [46], and processing speed and attention in MS [41]. There are also ongoing studies evaluating the effects of internet e-training interventions on spasticity [40], and on general motor performances in MS patients [38]. A clinical trial is currently evaluating the role of social networks to improve adherence to exercise programs [42].
Two different studies are currently implementing tailored robot-based rehabilitation interventions. In particular, authors are testing the software for a weight support robot for gait training [39], and for a self-adapting robot for arm training in MS [37].
There are ongoing clinical trials studying feasibility and effects of web-based psychotherapy in MS. The POWER@MS1 study, a randomised controlled clinical trial, is evaluating the effects of a 2-year online behavioural intervention on DMT decision making, disease management and lifestyle, and on subsequent changes in inflammatory activity of MS [43]. The iSLEEPms study, a pilot randomised controlled trial, is evaluating the effects of a 4-week online cognitive-behavioural intervention on MS patients with sleep disorders [44].
3.2. Standardising Previous Outcome Measures
We found eight studies out of 35 (23% of included studies) aiming to standardise and improve reliability of outcome measures already used in clinical trials, among which two are already published. In particular, digital technology has been used to standardise and improve reliability of the EDSS, and, more in general, of composite neurological and cognitive examination in MS.
Looking at the EDSS, one study is comparing conventional EDSS (paper version) with a new EDSS app [52]. In addition, in the EXPAND clinical trial, the EDSS, after being performed on site using Neurostatus E-Scoring App on an iPad (a companion app for this treatment development), was submitted online for remote revision. The Neurostatus E-Scoring App on iPad improved EDSS inter-rater reliability (with a reduction of inconsistencies from 32% to 1.5% during study conduction), and time spent in data collection (around 10 min per assessment) [71].
Looking at composite outcome measures, Rhodes and colleagues showed that the Multiple Sclerosis Performance Test (MSPT), a digital assessment tool built upon the previously-validated MSFC, allows the collection of standardised, quantitative, and clinically meaningful data in a clinical setting for individual assessments of patients with MS [49,72]. Similarly, in a sub-study of the phase 2 clinical trial of lisinopril in MS, conventional in-person MSFC has been compared with the mobile MSFC (a companion app for this treatment development), which is completed from the participants’ home using remote sensing technology and video conferencing [48].
Looking at upper and lower limb motor function, accelerometers on armbands have been compared with conventional measures of hand dexterity and ambulation in MS (e.g., 9HPT and T25WT, respectively) [50]. In another study, a newly developed App which performs hand dexterity tests is compared with the “Arm Function in Multiple Sclerosis Questionnaire”, a unidimensional 31-item questionnaire for measuring arm function in patients with MS, and with conventional hand exercises [51].
Finally, other studies have been evaluating EDSS-equivalent digital technology tools. In a proof-of-concept clinical trial, Midaglia and colleagues showed high adherence and satisfaction rates for the FLOODLIGHT test battery, including both active tests and passive monitoring (sensor-based gait and mobility recording), using smartphones and smartwatches [20]. Another study is assessing the efficacy of the BeCare app on smartphones to perform routine assessments of neurological functioning in subjects with MS; the BeCare app assigns EDSS scores obtained through multimodal information analysed with artificial intelligence techniques [54]. The same app is also been used to evaluate different neurological domains separately (cognition, afferent visual functioning, motor functioning, fine motor functioning, coordination, gait and endurance) [53].
3.3. Detection of Invisible Clinical and Treatment Features
We found 17 studies out of 35 (48% of included studies) aiming to improve detection of invisible clinical and treatment features based on digital technology, among which four are already published. In particular, digital technology has been used for daily monitoring of symptoms that would be otherwise difficult to detect, of treatment adherence and side effects, and for self-assessment purposes.
Looking at the invisible symptoms of MS, a number of ongoing studies are evaluating mobile Apps for daily monitoring of balance, falls [47,61], gait, fatigue and mood [57,58]. In a clinical trial designed to evaluate the effects of ginseng on MS-related fatigue, the authors developed the Real-Time Digital Fatigue Score (a companion app for this treatment development). This tool collects fatigue severity on a visual analogic scale (ranging from 0 to 10), through alarmed wristwatch device, worn by the patient, to measure and digitally record real-time experience of fatigue. Though the clinical trial failed to show significant treatment effect, the Real-Time Digital Fatigue Score has increased the statistical power for fatigue measurement [55].
Looking at treatment monitoring, in an already concluded clinical trial, the PROmyBETA App (a companion app for this treatment) was connected to electronic auto-injectors, and showed improved persistence, compliance, and adherence to interferon-beta1b over 6 months [59,65]. More recently, the PROmyBETA App has been integrated with a cognitive training tool designed with gaming elements, which could further contribute to improvements in interferon-beta1b usage [68]. In another ongoing trial, an electronic measure of needle disposal is used to measure adherence over two years for patients using injection therapy with glatiramer acetate [63]. Looking at oral medications (e.g., fingolimod, dimethyl fumarate, or teriflunomide), there is an ongoing trial evaluating an app connected to electronic pill bottles for monitoring medication usage and adherence over 90 days (time of bottle disposal) [70]. An electronic diary for MS patients, with medical data, patient reports, medication diaries and reminders, collected within a year of regular care, is under evaluation to measure and possibly improve adherence to different DMTs [60]. In line with this, Defer and colleagues showed that patients using the My eReport France® App were more precise in reporting on adverse drug reactions, when compared with spontaneous patient reports [64]. Another study is evaluating the impact of nurse training on MS patients in improving reports through this app [69]. However, in another study including patients affected by different neurological diseases and different behavioural and lifestyle interventions, the authors found no significant difference over 6 weeks between a paper and electronic diary for monitoring adherence to behavioural and lifestyle suggestions given to the patient in a clinical setting [73].
Looking at self-assessment tools, one study is currently evaluating a mobile program called Digital Self-Assessment for Multiple Sclerosis, composed of walking, coordination, attention, and visual tests [66]. This app is currently tested also to evaluate the association with disability measured by neurologist on MSFC [62]. Similarly, in a large study including 633 progressive MS patients treated with Ocrelizumab, authors are using a digital tool (SymptoMScreen, (a companion app for this study)) to measure walking, spasticity, cognition, vision, fatigue, hand function, pain, sensory, bladder control, balance and depression/anxiety [67].
New outcome measures could come also from nanotechnologies, which could detect new respiratory biomarkers of inflammation and neurodegeneration of MS through an artificial olfactory system [56].
4. Discussion
In the present systematic review, we showed how clinical trials in MS have been leveraging digital technology to improve delivery, adherence and effectiveness of motor rehabilitation and psychotherapy, to standardise collection and interpretation of already existing outcome measures, and to detect MS clinical and treatment features that would otherwise remain invisible in clinical practice, also with the development of new outcome measures. With a number of digital technology tools being developed, the importance of people with MS has emerged, in order to build the technology around the patients, rather than having them fit into the technology.
Digital technology has been allowing remote interventions in MS, with specific regard to rehabilitation and psychotherapy. Rehabilitation has especially benefited from digital technology, with unprecedented possibilities of remote delivery and improved adherence, while maintaining the effectiveness of conventional in-person rehabilitation [74,75]. For instance, exergames in virtual reality settings combine physical exercise with leisure components of games [10]. Evidence for the use of exergames in MS will progressively emerge in the upcoming years, with a number of ongoing studies evaluating its effects on motor function [45], fatigue [46], cognition [41], and spasticity [40]. Looking at previous positive results in Parkinson’s disease and stroke [75,76,77], we expect that exergames in virtual reality settings will become standard tool for remote exercising in MS. Psychotherapy could also be delivered remotely and, for instance, there are ongoing clinical trials in MS evaluating the effectiveness of online cognitive-behavioural interventions on disease activity [44], and symptoms (e.g., sleep disturbances) [44].
The EDSS has been used in more than 120 phase 2 and 3 clinical trials in MS, but with very few medications showing treatment effect [27]. The possibility of acquiring the EDSS using digital tools is already well established [27,52]. Recently, in the EXPAND clinical trial, the EDSS was performed on site, using Neurostatus E-Scoring App on iPad, and, then, submitted online for centralized revision, with improved inter-rater reliability, possibly contributing to the positive results of clinical trial medication (siponimod) on EDSS progression and subsequent approval for secondary progressive MS [71]. In addition, digital technology could further refine disability outcome measures by combining the EDSS with more precise digital measures of upper and lower limb motor function (e.g., mobile Apps for hand dexterity tests, accelerometers, sensing technology, video conferencing, etc.) [48,50,51]. Thus, digital technology is ultimately enabling the development of combined outcome measures collecting standardised, quantitative, and clinically-meaningful data in a clinical setting for individual assessments of patients with MS, which could be easily moved from clinical trials to clinical practice [49,72]. Of note, in the case of the FLOODLIGHT test battery, which includes both active tests and passive monitoring, approximately 90% of patients wanted to see the results of their tests in real-time [20], suggesting patients were actually engaged.
Artificial intelligence and machine learning techniques will further contribute to improve the standard of care in MS. For instance, the BeCare app uses multimodal information analysed with artificial intelligence techniques to assign EDSS scores [54], and to separately evaluate different neurological domains (cognition, afferent visual functioning, motor functioning, fine motor functioning, coordination, gait and endurance), thus providing a detailed neurological examination and overcoming the weight lower limbs have on the EDSS [53]. Also, robot-based rehabilitation, which is already part of clinical practice [78], will further benefit from software improvements to tailor rehabilitation programs based on patient needs [37,39,78,79].
Patients’ empowerment has been a cornerstone in designing clinical trials with clinically meaningful outcome measures [27], and the possibility of real-time monitoring of MS patients has shed light on the most invisible aspects of the disease. There are a number of ongoing clinical trials evaluating mobile apps for daily monitoring of balance, falls [47,61], gait, fatigue and mood, ultimately improving the measurements and increasing the statistical power [55,57,58]. Multi-dimensional data collection of different MS disabilities through a mobile app could refine the quality and accuracy of knowledge on the disease progression, with improved standard-of-care of MS patients [66]. Though invisible symptoms are common to many neurological and non-neurological diseases (e.g., fatigue) [80], homogeneous populations and targeted interventions should be preferred.
Non-adherence to DMTs is common, and is associated with higher risk of relapses and disability progression [81,82,83]. However, interventions to foster adherence in MS are costly, time-consuming, and difficult to evaluate in clinical practice [84,85]. Different studies have applied digital technology to auto-injectors [59,65], needle disposal systems [63], pill bottles [70], and diaries for monitoring DMT usage, adherence, and side effects [60,64].
Notwithstanding recent advances in digital technology in clinical trials (and daily life), there are limitations preventing the full applicability to clinical trials and clinical practice. First, digital technology access and use can be different between countries, due to cultural background, availability of high-speed connection, and trust in healthcare professionals [86]. Age is conventionally considered a limitation for digital technology, and, in line with this, most digital technologies in neurology have been applied in diseases affecting young adults, such as MS [87]. However, clinical trials we have hereby described included patients at any age, suggesting age-related limitations have been at least in part overcome [88]. Thus, a broader applicability would be very important for neurological diseases occurring in more advanced life stages (e.g., Parkinson’s disease, Alzheimer’s dementia), which could equally benefit from digital technology to detect their frequently subtle progression. Overall, clinical trial design should consider the actual possibility of using digital technology, along with possible bias coming from it (e.g., patients being aware of potential side effects of tested medications are more likely to present with placebo effect) [89]. The correct use of digital technology may be affected by the neurological condition itself (e.g., motor disability, visual impairment, psychiatric comorbidities, cognitive dysfunction) [88], thus limiting its applicability. The possibility of training a caregiver or a nurse to help with remote digital technology should be considered, though it has been tested only in one clinical trial [69]. There are financial limitations to the use of digital technology. Developing an app is relatively expensive, and requires a multi-step approach (i.e., device selection, app production, validation, data transfer) [90]. Most apps have been developed by private agencies that apply a fee, and, thus, at the moment, can only be used in selected population. Additional limitations are related to privacy and overall safety [89]. Finally, regulatory agencies do not frequently allow the use of outcome measures based on digital technology. The Food and Drug Administration has currently implemented a Digital Health Innovation Action Plan, which hopefully will enhance the potential of digital technology in clinical trials [91,92]. Similarly, the European Union has supported the Digital Transformation of Health and Care (Digicare), to develop a secure, flexible, and decentralised digital health infrastructure [93]. These initiatives will ultimately allow the standardisation of digital technology in clinical trials, observational studies, and clinical practice, enhancing its potentials while overcoming potential limitations [94].
Our systematic review has a number of limitations. Considering the relatively small number of included studies, we did not perform preliminary quality control, but only checked consistency with study objectives. Databases we have used (pubmed.gov, 26 May 2021, and clinicaltrials.gov, 26 May 2021) were not specifically organised for the topic of our review (e.g., absence of specific Medical Subject Headings), possibly limiting the number of included studies. Also, we need to acknowledge that a number of digital technology tools have been developed, but not included in clinical trials [93,95,96,97,98,99], and that digital health is a very dynamic and quickly evolving topic, with many studies being published after our research was carried out; thus, we have definitely missed some studies using digital technology in MS. However, we specifically focused on concluded and ongoing clinical trials (which are more likely to be translated to clinical practice) [27], while a full review of digital technology in MS should be evaluated in other studies.
5. Conclusions
With increasing numbers of MS patients using digital tools, especially following the COVID19 pandemic [100,101], the neurological and cognitive examination will be progressively shifting from paper-based on site, to internet-based on site, and, in the future, to internet-based at home. The latter could be particularly relevant since it allows continuous acquisition of real-world detailed outcome measures, moving beyond the limited snapshot of the neurological deficits assessed in clinical setting. As such, thanks to artificial intelligence algorithms, we could be able to detect relapses and disability automatically. Similarly, invisible symptoms of MS such as fatigue, cognitive impairment or anxiety/depression may be detected using digital technology, with the definition of totally new outcome measures, directly collected by patients in their real life, thus improving patients’ empowerment and participation into clinical trials.
MS is characterised by multiple symptoms (e.g., motor, sensory, cognitive, etc.) in common with other neurological disorders, and, thus, digital technology could be easily translated to other diseases. For instance, some clinical trials described above have already included patients with MS along with patients suffering from Parkinson’s disease, essential tremor, fibromyalgia, spinal cord injury, and stroke [37,39,73]. While there is no current digital technology standard in MS trials and research, over the next years, validation towards hard endpoints (i.e., EDSS 6.0 or EDSS 8.0) will possibly lead to a new universal language for monitoring disease outcomes and testing interventions.
In conclusion, digital technology is part of MS interventions and is widely used for cognitive and motor rehabilitation, with exergames, e-training, and robot-assisted exercises. Previously existing outcome measures are now collected using digital tools, in a reduced time frame, and with improved sensitivity. Specific platforms are available for the real-time detection of treatment adherence and side effects, and of symptoms which would have remained otherwise invisible, thanks to patients’ engagement and self-assessment. Of note, digital technology has been used in MS patients at any age, and with different clinical subtypes, suggesting it is able to tackle the spectrum of MS symptoms. Independently of its final goal, digital technology can develop a deep cultural change in clinical trial design and conduction, through tailoring studies and data collection on MS patients and positioning them at the very centre of the trial.
Author Contributions
Conceptualization, M.D.A., L.L. and M.M.; methodology, M.D.A., A.C., V.B.M. and M.M.; formal analysis, M.D.A., A.C., M.P. and R.L.; data curation, M.D.A., A.C., M.P. and R.L.; writing—original draft preparation, M.D.A., L.L. and M.M.; writing—review and editing, A.C., M.P., R.L. and V.B.M.; visualization, M.D.A. and M.P.; supervision, V.B.M. and M.M. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Bill F. Foundation MG. Global, regional, and national burden of multiple sclerosis 1990–2016: A systematic analysis for the Global Burden of Disease Study. Lancet Neurol. 2016;2019:269–285. doi: 10.1016/S1474-4422(18)30443-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Thompson A.J., Baranzini S.E., Geurts J., Hemmer B., Ciccarelli O. Multiple sclerosis. Lancet. 2018;391:1622–1636. doi: 10.1016/S0140-6736(18)30481-1. [DOI] [PubMed] [Google Scholar]
- 3.Giunti G., Fernández E.G., Zubiete E.D., Romero O.R. Supply and Demand in mHealth Apps for Persons with Multiple Sclerosis: Systematic Search in App Stores and Scoping Literature Review. JMIR mHealth uHealth. 2018;6:e10512. doi: 10.2196/10512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lorefice L., Fenu G., Frau J., Coghe G., Marrosu M.G., Cocco E. The impact of visible and invisible symptoms on employment status, work and social functioning in Multiple Sclerosis. Work. 2018;60:263–270. doi: 10.3233/WOR-182682. [DOI] [PubMed] [Google Scholar]
- 5.Marziniak M., Brichetto G., Feys P., Meyding-Lamadé U., Vernon K., Meuth S.G. The Use of Digital and Remote Communication Technologies as a Tool for Multiple Sclerosis Management: Narrative Review. JMIR Rehabilit. Assist. Technol. 2018;5:e5. doi: 10.2196/rehab.7805. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Taylor C.B., Ruzek J.I., Fitzsimmons-Craft E., Sadeh-Sharvit S., Topooco N., Weissman R.S., Eisenberg D., Mohr D., Graham A., Jacobi C., et al. Using Digital Technology to Reduce the Prevalence of Mental Health Disorders in Populations: Time for a New Approach. J. Med. Internet Res. 2020;22:e17493. doi: 10.2196/17493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rossen S., Kayser L., Vibe-Petersen J., Christensen J.F., Ried-Larsen M. Cancer Survivors’ Receptiveness to Digital Technolo-gy-Supported Physical Rehabilitation and the Implications for Design: Qualitative Study. J. Med. Int. Res. 2020;22:e15335. doi: 10.2196/15335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lee Y., Raviglione M.C., Flahault A. Use of Digital Technology to Enhance Tuberculosis Control: Scoping Review. J. Med. Internet Res. 2020;22:e15727. doi: 10.2196/15727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.World Health Organization (WHO) Classification of Digital Health Interventions v1.A Shared Language to Describe the Uses of Digital Technology for Health. WHO; Rome, Italy: 2018. [Google Scholar]
- 10.Lavorgna L., Brigo F., Moccia M., Leocani L., Lanzillo R., Clerico M., Abbadessa G., Schmierer K., Solaro C., Prosperini L., et al. e-Health and multiple sclerosis: An update. Mult. Scler. J. 2018;24:1657–1664. doi: 10.1177/1352458518799629. [DOI] [PubMed] [Google Scholar]
- 11.Haase R., Schultheiss T., Kempcke R., Thomas K., Ziemssen T. Use and Acceptance of Electronic Communication by Patients with Multiple Sclerosis: A Multicenter Questionnaire Study. J. Med. Internet Res. 2012;14:e135. doi: 10.2196/jmir.2133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Thomas S., Pulman A., Thomas P., Collard S., Jiang N., Dogan H., Smith A.D., Hourihan S., Roberts F., Kersten P., et al. Digitizing a Face-to-Face Group Fatigue Management Program: Exploring the Views of People with Multiple Sclerosis and Health Care Professionals Via Consultation Groups and Interviews. JMIR Form. Res. 2019;3:e10951. doi: 10.2196/10951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Simblett S., Matcham F., Curtis H., Greer B., Polhemus A., Novák J., Ferrao J., Gamble P., Hotopf M., Narayan V., et al. Patients’ Measurement Priorities for Remote Measurement Technologies to Aid Chronic Health Conditions: Qualitative Analysis. JMIR mHealth uHealth. 2020;8:e15086. doi: 10.2196/15086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Thirumalai M., Rimmer J.H., Johnson G., Wilroy J., Young H.-J., Mehta T., Lai B. TEAMS (Tele-Exercise and Multiple Sclerosis), a Tailored Telerehabilitation mHealth App: Participant-Centered Development and Usability Study. JMIR mHealth uHealth. 2018;6:e10181. doi: 10.2196/10181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kayser L., Karnoe A., Furstrand D., Batterham R., Christensen K.B., Elsworth G., Osborne R.H. A Multidimensional Tool Based on the eHealth Literacy Framework: Development and Initial Validity Testing of the eHealth Literacy Questionnaire (eHLQ) J. Med. Internet Res. 2018;20:e36. doi: 10.2196/jmir.8371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Zhou L., Parmanto B. Reaching People with Disabilities in Underserved Areas Through Digital Interventions: Systematic Review. J. Med. Internet Res. 2019;21:e12981. doi: 10.2196/12981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Van Kessel K., Babbage D.R., Reay N., Miner-Williams W.M., Kersten P. Mobile Technology Use by People Experiencing Multiple Sclerosis Fatigue: Survey Methodology. JMIR mHealth uHealth. 2017;5:e6. doi: 10.2196/mhealth.6192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Green B.M., Van Horn K.T., Gupte K., Evans M., Hayes S., Bhowmick A. Adaptive Engagement and Support Model for People with Chronic Health Conditions: Using Combined Content Analysis to Assess Online Health Communities. J. Med. Internet Res. 2020 doi: 10.2196/17338. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.D’Haeseleer M., Eelen P., Sadeghi N., D’Hooghe M.B., Van Schependom J., Nagels G. Feasibility of real-time internet-based tel-econsultation in patients with multiple sclerosis: Interventional pilot study. J. Med. Internet Res. 2020;22:e18178. doi: 10.2196/18178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Midaglia L., Mulero P., Montalban X., Graves J., Hauser S.L., Julian L., Baker M., Schadrack J., Gossens C., Scotland A., et al. Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People with Multiple Sclerosis: Nonrandomized Interventional Feasibility Study. J. Med. Internet Res. 2019;21:e14863. doi: 10.2196/14863. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Amato M.P., Fratiglioni L., Groppi C., Siracusa G., Amaducci L. Interrater Reliability in Assessing Functional Systems and Dis-ability on the Kurtzke Scale in Multiple Sclerosis. Arch. Neurol. 1988;45:746–748. doi: 10.1001/archneur.1988.00520310052017. [DOI] [PubMed] [Google Scholar]
- 22.Hirst C.L., Ingram G., Pickersgill T.P., Robertson N.P. Temporal evolution of remission following multiple sclerosis relapse and predictors of outcome. Mult. Scler. J. 2012;18:1152–1158. doi: 10.1177/1352458511433919. [DOI] [PubMed] [Google Scholar]
- 23.Moccia M., De Stefano N., Barkhof F. Imaging outcome measures for progressive multiple sclerosis trials. Mult. Scler. J. 2017;23:1614–1626. doi: 10.1177/1352458517729456. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hohol M.J., Orav E.J., Weiner H.L. Disease steps in multiple sclerosis: A longitudinal study comparing Disease Steps and EDSS to evaluate disease progression. Mult. Scler. J. 1999;5:349–354. doi: 10.1177/135245859900500508. [DOI] [PubMed] [Google Scholar]
- 25.Cadavid D., Tang Y., O’Neill G. Responsiveness of the Expanded Disability Status Scale (EDSS) to disease progression and therapeutic intervention in progressive forms of multiple sclerosis. Rev. Neurol. 2010;51:321–329. [PubMed] [Google Scholar]
- 26.Saccà F., Costabile T., Carotenuto A., Lanzillo R., Moccia M., Pane C., Russo C.V., Barbarulo A.M., Casertano S., Rossi F., et al. The EDSS integration with the Brief International Cognitive Assessment for Multiple Sclerosis and orientation tests. Mult. Scler. 2017;23:1289–1296. doi: 10.1177/1352458516677592. [DOI] [PubMed] [Google Scholar]
- 27.Tur C., Moccia M., Barkhof F., Chataway J., Sastre-Garriga J., Thompson A.J., Ciccarelli O. Assessing treatment outcomes in multiple sclerosis trials and in the clinical setting. Nat. Rev. Neurol. 2018;14:75–93. doi: 10.1038/nrneurol.2017.171. [DOI] [PubMed] [Google Scholar]
- 28.Jongen P.J., Kremer I.E., Hristodorova E., Evers S.M., Kool A., Van Noort E.M., Hiligsmann M., Haase R., Lavorgna L. Adherence to Web-Based Self-Assessments in Long-Term Direct-to-Patient Research: Two-Year Study of Multiple Sclerosis Patients. J. Med. Internet Res. 2017;19:e249. doi: 10.2196/jmir.6729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Khurana V., Sharma H., Afroz N., Callan A., Medin J. Patient-reported outcomes in multiple sclerosis: A systematic comparison of available measures. Eur. J. Neurol. 2017;24:1099–1107. doi: 10.1111/ene.13339. [DOI] [PubMed] [Google Scholar]
- 30.D’Amico E., Haase R., Ziemssen T. Review: Patient-reported outcomes in multiple sclerosis care. Mult. Scler. Relat. Disord. 2019;33:61–66. doi: 10.1016/j.msard.2019.05.019. [DOI] [PubMed] [Google Scholar]
- 31.Ontaneda D., Fox R.J., Chataway J. Clinical trials in progressive multiple sclerosis: Lessons learned and future perspectives. Lancet Neurol. 2015;14:208–223. doi: 10.1016/S1474-4422(14)70264-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Graves J.S., Montalban X. Biosensors to monitor MS activity. Mult. Scler. J. 2020;26:605–608. doi: 10.1177/1352458519888178. [DOI] [PubMed] [Google Scholar]
- 33.Inojosa H., Schriefer D., Ziemssen T. Clinical outcome measures in multiple sclerosis: A review. Autoimmun. Rev. 2020;19:102512. doi: 10.1016/j.autrev.2020.102512. [DOI] [PubMed] [Google Scholar]
- 34.Barkhof F., Scheltens P., Frequin S., Nauta J., Tas M., Valk J., Hommes O. Relapsing-remitting multiple sclerosis: Sequential enhanced MR imaging vs clinical findings in determining disease activity. AJR Am. J. Roentgenol. 1992;159:1041–1047. doi: 10.2214/ajr.159.5.1414773. [DOI] [PubMed] [Google Scholar]
- 35.Baldassari L.E., Planchon S.M., Bermel R.A., Nakamura K., Fisher E., Feng J., Sakaie K.E., Ontaneda D., Cohen J.A. Serum neuro-filament light chain concentration in a phase 1/2 trial of autologous mesenchymal stem cell transplantation. Mult. Scler. J. Exp. Transl. Clin. 2019;5:205521731988719. doi: 10.1177/2055217319887198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Kuhle J., Disanto G., Lorscheider J., Stites T., Chen Y., Dahlke F., Francis G., Shrinivasan A., Radue E.-W., Giovannoni G., et al. Fingolimod and CSF neurofilament light chain levels in relapsing-remitting multiple sclerosis. Neurology. 2015;84:1639–1643. doi: 10.1212/WNL.0000000000001491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. NCT01918748. Effect of I-TRAVLE Training on Arm Function in MS and Chronic Stroke Patients. US National Library of Medicine; Bethesda, MD, USA: 2013. [Google Scholar]
- 38. NCT02771652. Effect of an Internet-Based at Home Physical Training Protocol on Quality of Life, Fatigue, Functional Performance, Aerobic Capacity and Muscle Strength in Multiple Sclerosis Patients (ms-intakt) US National Library of Medicine; Bethesda, MD, USA: 2016. [Google Scholar]
- 39. NCT03458169. LEAP a New Overground Body Weight Support. Robot.: Usability Trial (LEAP) US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 40. NCT03548974. Internet-Based MOTOmed Exercise to Reduce Spasticity and Improve Physical Function in Persons with Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2018. [Google Scholar]
- 41. NCT03569618. Digital Cognition in Multiple Sclerosis (DigCog) University of California; US National Library of Medicine; Bethesda, MD, USA: 2020. [Google Scholar]
- 42. NCT03829267. eFIT: An Internet-based Intervention to Increase Physical Activity in Persons with MS (eFIT) US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 43. NCT03968172. Interactive Web Platform for EmPOWERment in Early Multiple Sclerosis (POWER@MS1) US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 44. NCT04147052. iSLEEPms: An Internet-Delivered Intervention for Sleep Disturbance in Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 45. NCT04212689. Virtual Reality Approach in Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 46. NCT04244214. Pilot Study for the Evaluation of the More Stamina in Persons with Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2020. [Google Scholar]
- 47. NCT01829776. Pilot Study of Free from Falls Program in Multiple Sclerosis (FFF) US National Library of Medicine; Bethesda, MD, USA: 2013. [Google Scholar]
- 48. NCT02369926. Endpoint Calibration for a Phase 2 Study of Lisinopril in Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2015. [Google Scholar]
- 49.Rhodes J.K., Schindler D., Rao S.M., Venegas F., Bruzik E.T., Gabel W., Williams J.R., Phillips G.A., Mullen C.C., Freiburger J.L., et al. Multiple Sclerosis Performance Test: Technical Development and Usability. Adv. Ther. 2019;36:1741–1755. doi: 10.1007/s12325-019-00958-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. NCT03369171. Wearable Biosensor to Track and Quantify Limb Dysfunction in Multiple Sclerosis Patients (MYO) US National Library of Medicine; Bethesda, MD, USA: 2017. [Google Scholar]
- 51. NCT03369470. App Based Dexterity Training in Multiple Sclerosis (AppDext) US National Library of Medicine; Bethesda, MD, USA: 2017. [Google Scholar]
- 52. NCT03586635. Automated EDSS Score Calculation by Smartphone Application (Easy EDSS) US National Library of Medicine; Bethesda, MD, USA: 2018. [Google Scholar]
- 53. NCT04281160. Patient Centered Outcomes Analysis for MS Using a Mobile Application. US National Library of Medicine; Bethesda, MD, USA: 2020. [Google Scholar]
- 54. NCT04288011. Validation of the BeCare Multiple Sclerosis Assessment App. US National Library of Medicine; Bethesda, MD, USA: 2020. [Google Scholar]
- 55.Kim E., Lovera J., Schaben L., Melara J., Bourdette D., Whitham R. Novel method for measurement of fatigue in multiple sclerosis: Real-Time Digital Fatigue Score. J. Rehabil. Res. Dev. 2010;47:477–484. doi: 10.1682/JRRD.2009.09.0151. [DOI] [PubMed] [Google Scholar]
- 56. NCT01206023. Applications of Nanotechnology in Multiple Sclerosis by Respiratory Samples (MS-NANOSE) US National Library of Medicine; Bethesda, MD, USA: 2010. [Google Scholar]
- 57. NCT01233245. BetaPlus Survey—Observational Study to Assess. Drug Adherence in Patients with Multiple Sclerosis after Conversion to Betaferon® by Using Elements of the BetaPlus Program. US National Library of Medicine; Bethesda, MD, USA: 2010. [Google Scholar]
- 58. NCT01235455. Portuguese Observational Survey to Assess. Drug Adherence in Patients with Multiple Sclerosis after Conversion to Betaferon by Using Elements of the BetaPlus Program—Nurse Support. Auto-injectors (POR-BetaPlus) US National Library of Medicine; Bethesda, MD, USA: 2010. [Google Scholar]
- 59.Salimzadeh Z., Damanabi S., Kalankesh L.R., Ferdousi R. Mobile Applications for Multiple Sclerosis: A Focus on Self-Management. Acta Inform. Med. 2019;27:12–18. doi: 10.5455/aim.2019.27.12-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60. NCT01954823. Use of an Electronic Diary by People with Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2013. [Google Scholar]
- 61. NCT02583386. Comprehensive Fall Prevention and Detection in Multiple Sclerosis (FFF) US National Library of Medicine; Bethesda, MD, USA: 2015. [Google Scholar]
- 62. NCT02814487. MSFC versus DAM. A Smartphone Application for Multiple Sclerosis Self-Assessment (DAM) US National Library of Medicine; Bethesda, MD, USA: 2016. [Google Scholar]
- 63. NCT01419301. Mindfulness Based Stress Reduction in Multiple Sclerosis (MS) US National Library of Medicine; Bethesda, MD, USA: 2011. [Google Scholar]
- 64.Defer G., Le Caignec F., Fedrizzi S., Montastruc F., Chevanne D., Parienti J.-J., Peyro-Saint-Paul L. Dedicated mobile application for drug adverse reaction reporting by patients with relapsing remitting multiple sclerosis (Vigip-SEP study): Study protocol for a randomized controlled trial. Trials. 2018;19:174. doi: 10.1186/s13063-018-2560-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Limmroth V., Hechenbichler K., Müller C., Schürks M. Assessment of Medication Adherence Using a Medical App Among Patients with Multiple Sclerosis Treated with Interferon Beta-1b: Pilot Digital Observational Study (PROmyBETAapp) J. Med. Internet Res. 2019;21:e14373. doi: 10.2196/14373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66. NCT03148938. Digitalization of Neurofunctional Tests via a Mobile Application DAMS for Multiple Sclerosis Patients (MSCopilot) US National Library of Medicine; Bethesda, MD, USA: 2017. [Google Scholar]
- 67. NCT03523858. A Study to Evaluate Ocrelizumab Treatment in Participants with Progressive Multiple Sclerosis (CONSO-NANCE) US National Library of Medicine; Bethesda, MD, USA: 2018. [Google Scholar]
- 68. NCT03808142. PROmyBETAappGame: A Study to Learn. More about the Medication Usage & Patient Reported Outcomes via the myBETAapp and to Find Out More about the Usage of Game Principles and Game Design Elements (Gamification) in Medical Care of Patients with Multiple Scle. US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 69. NCT04116424. VIGIP-SEP2: Evaluation of the Impact of the Training of Patients by a Nurse on the Adverse Drug Reaction Reporting by RRMS Patient via a Mobile Application: Randomized Real-Life Study (VIGIP-SEP2) US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 70. NCT04130256. Electronic Pill Bottle Monitoring to Promote Medication Adherence for People with Multiple Sclerosis. US National Library of Medicine; Bethesda, MD, USA: 2019. [Google Scholar]
- 71.D’Souza M., Yaldizli Ö., John R., Vogt D.R., Papadopoulou A., Lucassen E., Menegola M., Andelova M., Dahlke F., Schnyder F., et al. Neurostatus e-Scoring improves consistency of Expanded Disability Status Scale assessments: A proof of concept study. Mult. Scler. J. 2016;23:597–603. doi: 10.1177/1352458516657439. [DOI] [PubMed] [Google Scholar]
- 72.Meyer-Moock S., Feng Y.-S., Maeurer M., Dippel F.-W., Kohlmann T. Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis. BMC Neurol. 2014;14:58. doi: 10.1186/1471-2377-14-58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Plow M., Golding M. Using mHealth Technology in a Self-Management Intervention to Promote Physical Activity Among Adults with Chronic Disabling Conditions: Randomized Controlled Trial. JMIR mHealth uHealth. 2017;5:e185. doi: 10.2196/mhealth.6394. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Leocani L., Diserens K., Moccia M., Caltagirone C., Neurology-Ean N.S.P.O.T.E.A.O., Neurorehabilitation Scientific Panel of the European Academy of Neurology—EAN Disability through COVID-19 pandemic: Neurorehabilitation cannot wait. Eur. J. Neurol. 2020;27 doi: 10.1111/ene.14320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Isernia S., Pagliari C., Jonsdottir J., Castiglioni C., Gindri P., Gramigna C., Palumbo G., Salza M., Molteni F., Baglio F. Efficiency and Patient-Reported Outcome Measures From Clinic to Home: The Human Empowerment Aging and Disability Program for Digi-tal-Health Rehabilitation. Front. Neurol. 2019;10:1206. doi: 10.3389/fneur.2019.01206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Taylor M.J.D., Griffin M. The use of gaming technology for rehabilitation in people with multiple sclerosis. Mult. Scler. J. 2015;21:355–371. doi: 10.1177/1352458514563593. [DOI] [PubMed] [Google Scholar]
- 77.Mura G., Carta M.G., Sancassiani F., Machado S., Prosperini L. Active exergames to improve cognitive functioning in neuro-logical disabilities: A systematic review and meta-analysis. Eur. J. Phys. Rehabil. Med. 2018;54:450–462. doi: 10.23736/S1973-9087.17.04680-9. [DOI] [PubMed] [Google Scholar]
- 78.Schwartz I., Meiner Z. Robotic-Assisted Gait Training in Neurological Patients: Who May Benefit? Ann. Biomed. Eng. 2015;43:1260–1269. doi: 10.1007/s10439-015-1283-x. [DOI] [PubMed] [Google Scholar]
- 79.Yue Z., Zhang X., Wang J. Hand Rehabilitation Robotics on Poststroke Motor Recovery. Behav. Neurol. 2017;2017:1–20. doi: 10.1155/2017/3908135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Moccia M., Picillo M., Erro R., Vitale C., Longo K., Amboni M., Santangelo G., Palladino R., Capo G., Orefice G., et al. Presence and progression of non-motor symptoms in relation to uric acid in de novo Parkinson’ s disease. Eur. J. Neurol. 2015;22:93–98. doi: 10.1111/ene.12533. [DOI] [PubMed] [Google Scholar]
- 81.Moccia M., Palladino R., Russo C., Massarelli M., Nardone A., Triassi M., Lugaresi A., Morra V.B. How many injections did you miss last month? A simple question to predict interferon β-1a adherence in multiple sclerosis. Expert Opin. Drug Deliv. 2015;12:1829–1835. doi: 10.1517/17425247.2015.1078789. [DOI] [PubMed] [Google Scholar]
- 82.Järvinen E., Multanen J., Atula S. Subcutaneous Interferon β-1a Administration by Electronic Auto-Injector is Associated with High Adherence in Patients with Relapsing Remitting Multiple Sclerosis in a Real-Life Study. Neurol. Int. 2017;9:8–10. doi: 10.4081/ni.2017.6957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Wicks P., Massagli M., Kulkarni A., Dastani H., Jacobs D., Jongen P. Use of an Online Community to Develop Patient-Reported Outcome Instruments: The Multiple Sclerosis Treatment Adherence Questionnaire (MS-TAQ) J. Med. Internet Res. 2011;13:e12. doi: 10.2196/jmir.1687. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Yeh E.A., Grover S.A., Powell V.E., Alper G., Banwell B.L., Edwards K., Gorman M., Graves J., Lotze T.E., Mah J.K., et al. Impact of an electronic monitoring device and behavioral feedback on adherence to multiple sclerosis therapies in youth: Results of a randomized trial. Qual. Life Res. 2017;26:2333–2349. doi: 10.1007/s11136-017-1571-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Moccia M., Loperto I., Lanzillo R., Capacchione A., Carotenuto A., Triassi M., Morra V.B., Palladino R. Persistence, adherence, healthcare resource utilisation and costs for interferon Beta in multiple sclerosis: A population-based study in the Campania region (southern Italy) BMC Health Serv. Res. 2020;20:1–8. doi: 10.1186/s12913-020-05664-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 86.Hone T., Palladino R., Filippidis F.T. Association of searching for health-related information online with self-rated health in the European Union. Eur. J. Public Health. 2016;26:748–753. doi: 10.1093/eurpub/ckw022. [DOI] [PubMed] [Google Scholar]
- 87.Moccia M., Brigo F., Tedeschi G., Bonavita S., Lavorgna L. Neurology and the Internet: A review. Neurol. Sci. 2018;39:981–987. doi: 10.1007/s10072-018-3339-9. [DOI] [PubMed] [Google Scholar]
- 88.Sharma A., Harrington R.A., McClellan M.B., Turakhia M.P., Eapen Z.J., Steinhubl S., Mault J.R., Majmudar M.D., Roessig L., Chandross K.J., et al. Using Digital Health Technology to Better Generate Evidence and Deliver Evidence-Based Care. J. Am. Coll. Cardiol. 2018;71:2680–2690. doi: 10.1016/j.jacc.2018.03.523. [DOI] [PubMed] [Google Scholar]
- 89.Benedetti F. Placebo Effects: From the Neurobiological Paradigm to Translational Implications. Neuron. 2014;84:623–637. doi: 10.1016/j.neuron.2014.10.023. [DOI] [PubMed] [Google Scholar]
- 90.Bergendorff R. Digital Devices in Clinical Trials Clinical and Medical Research. IPI Clin. Med. Res. 2018;10 [Google Scholar]
- 91.FDA . Digital Health Innovation Action Plan. FDA; Silver Spring, MD, USA: 2017. [Google Scholar]
- 92.Aarestrup F.M., Albeyatti A., Armitage W.J., Auffray C., Augello L., Balling R., Benhabiles N., Bertolini G., Bjaalie J.G., Black M., et al. Towards a European health research and innovation cloud (HRIC) Genome Med. 2020;12:1–14. doi: 10.1186/s13073-020-0713-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Tacchino A., Veldkamp R., Coninx K., Brulmans J., Palmaers S., Hämäläinen P., D’Hooge M., Vanzeir E., Kalron A., Brichetto G., et al. Cognitive-motor interference in people with Multiple Sclerosis: Design, development and testing of CMI-APP for dual task assessment and training. JMIR mHealth uHealth. 2019;8 doi: 10.2196/15344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Moccia M. Teleconsultation will replace most face-to-face interactions in the multiple sclerosis clinic—No. Mult. Scler. J. 2021;27:176–177. doi: 10.1177/1352458520968815. [DOI] [PubMed] [Google Scholar]
- 95.Van Dongen L., Westerik B., van der Hiele K., Visser L.H., Schoonheim M.M., Douw L., Twisk J.W.R., de Jong B.A., Geurts J.J.G., Hulst H.E. Introducing Multiple Screener: An unsupervised digital screening tool for cognitive deficits in MS. Mult. Scler. Relat. Disord. 2020;38:101479. doi: 10.1016/j.msard.2019.101479. [DOI] [PubMed] [Google Scholar]
- 96.Ziemssen T., Piani-Meier D., Bennett B., Johnson C., Tinsley K., Trigg A., Hach T., Dahlke F., Tomic D., Tolley C., et al. A Physician-Completed Digital Tool for Evaluating Disease Progression (Multiple Sclerosis Progression Discussion Tool): Validation Study. J. Med. Internet Res. 2020;22:e16932. doi: 10.2196/16932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97.Apolinário-Hagen J., Menzel M., Hennemann S., Salewski C. Acceptance of Mobile Health Apps for Disease Management Among People with Multiple Sclerosis: Web-Based Survey Study. JMIR Form. Res. 2018;2:e11977. doi: 10.2196/11977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98.Van Beek J.J.W., van Wegen E.E.H., Rietberg M.B., Nyffeler T., Bohlhalter S., Kamm C.P., Nef T., Vanbellingen T. Feasibility of a Home-Based Tablet App for Dexterity Training in Multiple Sclerosis: Usability Study. JMIR mHealth uHealth. 2020;8:e18204. doi: 10.2196/18204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99.Tacchino A., Veldkamp R., Coninx K., Brulmans J., Palmaers S., Hämäläinen P., D’hooge M., Vanzeir E., Kalron A., Brichetto G., et al. Design, Development, and Testing of an App for Dual-Task Assessment and Training Regarding Cognitive-Motor Interference (CMI-APP) in People with Multiple Sclerosis: Multicenter Pilot Study. JMIR mHealth uHealth. 2020;8:e15344. doi: 10.2196/15344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Moccia M., Lanzillo R., Morra V.B., Bonavita S., Tedeschi G., Leocani L., Lavorgna L., on behalf of the Digital Technologies Web and Social Media Study Group of the Italian Society of Neurology Assessing disability and relapses in multiple sclerosis on tele-neurology. Neurol. Sci. 2020;41:1369–1371. doi: 10.1007/s10072-020-04470-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 101.Moccia M., Palladino R., Falco A., Saccà F., Lanzillo R., Brescia Morra V. Google Trends: New evidence for seasonality of multiple sclerosis. J. Neurol. Neurosurg. Psychiatry. 2016;87:1028–1029. doi: 10.1136/jnnp-2016-313260. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.