Skip to main content
Annals of Clinical and Translational Neurology logoLink to Annals of Clinical and Translational Neurology
. 2020 Dec 22;8(2):308–320. doi: 10.1002/acn3.51236

Design of a virtual longitudinal observational study in Parkinson’s disease (AT‐HOME PD)

Ruth B Schneider 1,2,, Larsson Omberg 3, Eric A Macklin 4,5, Margaret Daeschler 6, Lauren Bataille 6, Shalini Anthwal 2, Taylor L Myers 2, Elizabeth Baloga 2, Sidney Duquette 2, Phil Snyder 3, Katherine Amodeo 1, Christopher G Tarolli 1,2, Jamie L Adams 1,2, Katherine F Callahan 7, Joshua Gottesman 6, Catherine M Kopil 6, Codrin Lungu 8, Alberto Ascherio 9, James C Beck 10, Kevin Biglan 1,11, Alberto J Espay 12, Caroline Tanner 13, David Oakes 14, Ira Shoulson 1,2,15, Dan Novak 10, Elise Kayson 1,2, Earl Ray Dorsey 1,2, Lara Mangravite 3, Michael A Schwarzschild 7, Tanya Simuni 16; the Parkinson Study Group AT‐HOME PD Investigators
PMCID: PMC7886038  PMID: 33350601

Abstract

Objective

The expanding power and accessibility of personal technology provide an opportunity to reduce burdens and costs of traditional clinical site‐centric therapeutic trials in Parkinson’s disease and generate novel insights. The value of this approach has never been more evident than during the current COVID‐19 pandemic. We sought to (1) establish and implement the infrastructure for longitudinal, virtual follow‐up of clinical trial participants, (2) compare changes in smartphone‐based assessments, online patient‐reported outcomes, and remote expert assessments, and (3) explore novel digital markers of Parkinson’s disease disability and progression.

Methods

Participants from two recently completed phase III clinical trials of inosine and isradipine enrolled in Assessing Tele‐Health Outcomes in Multiyear Extensions of Parkinson’s Disease trials (AT‐HOME PD), a two‐year virtual cohort study. After providing electronic informed consent, individuals complete annual video visits with a movement disorder specialist, smartphone‐based assessments of motor function and socialization, and patient‐reported outcomes online.

Results

From the two clinical trials, 226 individuals from 42 states in the United States and Canada enrolled. Of these, 181 (80%) have successfully downloaded the study’s smartphone application and 161 (71%) have completed patient‐reported outcomes on the online platform.

Interpretation

It is feasible to conduct a large‐scale, international virtual observational study following the completion of participation in brick‐and‐mortar clinical trials in Parkinson’s disease. This study, which brings research to participants, will compare established clinical endpoints with novel digital biomarkers and thereby inform the longitudinal follow‐up of clinical trial participants and design of future clinical trials.

Introduction

Parkinson’s disease (PD) is a progressive neurodegenerative disorder that affects nearly one million Americans. 1 Current treatment is symptomatic only; so far, there are no therapeutics that slow the progression of this debilitating disease despite multiple trials. One reason is that therapeutic development for long‐term indications in PD is limited by our current methods of measuring clinically meaningful disease progression. According to the Tufts Center for the Study of Drug Development, clinical trials for central nervous system disorders—like PD—are larger, longer, more expensive, and more likely to fail to demonstrate efficacy than for other disorders. 2 In addition, clinical trials are burdensome, routinely exclude large numbers of individuals based on geographic or social factors, and rely on subjective, insensitive, and episodic rating scales that generate conflicting, false, and uncertain signals of efficacy. 3 , 4 , 5 , 6 , 7

New approaches to clinical drug development for PD are needed. 8 Among these are virtual studies that bring research opportunities to participants rather than participants to research. Such studies, which are often site‐less or conducted from a single site, can enroll participants across large geographical areas, do so in relatively short periods of time, and are well received by participants. 9 , 10 , 11 In addition, a new generation of digital tools—including smartphones, wearable sensors, and passive in‐home monitors—allows frequent and objective assessments of PD, capturing the lived experience in real‐world settings. 12 , 13 , 14 A recent phase 1 clinical trial in PD demonstrated the feasibility and potential benefits of such an approach, including a more sensitive assessment of tremor and measurement of important outcomes, such as activity. 15 The 21st Century Cures Act, passed in 2016, called for the use of real‐world data, including that generated by patients in their home or from mobile devices, in regulatory decision making. 16 The law has led to multiple FDA guidance documents on the use of real‐world data and real‐world evidence. 16

The current COVID‐19 pandemic, which has impacted on‐going and new in‐person clinical PD research studies 17 highlights the need for such novel clinical research models and tools. On‐going PD research studies have faced challenges of how to dispense study drug, conduct safety and efficacy assessments, and collect biological specimens. The FDA has issued guidance recognizing that “ensuring the safety of the clinical trial participant is paramount” and that alternative methods for safety and efficacy assessments, such as virtual visits, should be considered. Meanwhile, remote studies such as an NIH‐funded, 250‐person observational study of LRRK2 carriers, 19 the online 45,000 plus person Fox Insight study, 20 a completely home‐based NIH‐funded randomized controlled PD trial (Trial of Parkinson’s and Zoledronic Acid [TOPAZ]; NCT03924414), 21 and the study described in this manuscript have been able to continue research operations seamlessly during the pandemic. These studies showcase research methods that are currently or could be employed in interventional studies.

In 2018, we launched a longitudinal virtual observational study in PD that enrolled participants from two phase III clinical trials of isradipine (STEADY‐PD III) and inosine (SURE‐PD3), ongoing at the time. 22 , 23 The study includes video visits with a movement disorder specialist, smartphone‐based assessments at home (mPower Progression study), and patient‐reported outcomes collected on an online platform (Fox Insight). The aims of the study are (1) to establish and implement the infrastructure for longitudinal, remote follow‐up of clinical trial cohorts, (2) to compare changes in participant‐driven outcomes (e.g., smartphone‐based assessments, web‐based surveys) and clinician‐driven (e.g., remote video assessments), and (3) to explore novel digital markers of PD disability and progression. The objective of this paper is to outline study methodology and provide guidance to other investigators on how to launch and implement virtual studies in PD especially in view of the COVID‐19 pandemic.

Methods

The study methods are reported in concordance with the STROBE statement. 24

Study design

AT‐HOME PD (Assessing Tele‐Health Outcomes in Multi‐year Extensions of Parkinson’s Disease Trials) is a 24‐month remote observational study of individuals who participated in the recently completed phase three clinical trials of isradipine (Efficacy of Isradipine in Early Parkinson Disease [STEADY‐PD III]; NCT02168842) 22 and inosine (Study of Urate Elevation in Parkinson's Disease, Phase 3 [SURE‐PD3]; NCT02642393). 23 As shown in Figure 1, the virtual study has three principal components: (1) annual video visits between research participants and a remotely located movement disorder specialist; (2) quarterly smartphone‐based assessments and passive data collection using the mPower 2.0 Parkinson’s smartphone application, 25 and (3) quarterly web‐based surveys through Fox Insight, an online clinical research study sponsored by The Michael J. Fox Foundation for Parkinson’s Research (MJFF). 20

Figure 1.

Figure 1

Overview of AT‐HOME PD Study.

Study setting

The study is a collaboration of the lead centers for the isradipine and inosine (Northwestern University and Massachusetts General Hospital, respectively) clinical trials, coordinating center (University of Rochester), nonprofit organizations supporting the smartphone and online research platforms (Sage Bionetworks and MJFF, respectively), and the Parkinson Study Group (PSG). The study is funded by the National Institute of Neurological Disorders and Stroke (NINDS, NCT 03538262). The virtual visits are conducted by the study team at the University of Rochester, a PSG‐credentialed clinical site. The study was reviewed and approved by the institutional review board at the University of Rochester.

Study participants

The key inclusion criteria were (a) current or past enrollment in the STEADY‐PD III or SURE‐PD3 clinical trials (participants had to have a diagnosis of early PD and not be on dopaminergic therapy at the time of enrollment into the parent studies, except that a monoamine oxidase‐B inhibitor was allowed at baseline in SURE‐PD3); the full eligibility criteria for respective studies have been previously published, 22 , 23 (b) prior consent to be contacted by the University of Rochester or direct contact with the coordinating site by a potential participant, (c) access to an internet‐enabled device that would support video visits, (d) existence of or willingness to create a Global Unique Identifier, which enables linking of participant data across multiple studies, (e) willingness and ability to provide informed consent, and (f) English fluency. Participation in video visits was required for all participants; participation in the smartphone and Fox Insight components were strongly encouraged but not mandatory. Participants without a suitable web camera were provided with an external web camera. For those choosing to participate in the smartphone component, possession of a suitable smartphone (iOS or Android) with an adequate data plan and cellular network access/signal or Wi‐Fi access was required. For racial and ethnic minorities without the necessary technological resources, we offered to reimburse the costs of standard internet access and/or a smartphone/data plan. The sole exclusion criterion was an inability to carry out study activities.

Study procedures and outcome measures

At the end of their respective studies, participants from the phase three clinical trials of isradipine and inosine who consented to be contacted for future research were approached by AT‐HOME PD study team members, through a registry of the parent studies. Participants were contacted via phone call, email, and/or mail regarding participation in the follow‐up study. Interested and eligible individuals were then emailed a link to an electronic informed consent form in Research Electronic Data Capture (REDCap; Nashville, TN), a secure, web‐based data management system.

The last STEADY‐PD III study visit occurred on November 20, 2018, and the majority of STEADY‐PD III participants were approached after completing all study activities. The last SURE‐PD3 study visit occurred on June 7, 2019, after a prespecified interim analysis demonstrated futility for the primary outcome. While our intention was to enroll SURE‐PD3 participants prior to their final study visit to allow the validation of remote motor assessments against in‐person assessments, early study closure curtailed these efforts and shortened the anticipated recruitment period for SURE‐PD3 participants.

Video visits

The video visits are conducted from the University of Rochester using secure video conferencing software from Zoom (San Jose, CA). Participants complete the video visits on a computer, tablet, or smartphone. After electronic consent is obtained, participants complete a survey on technology use in REDCap and a test video visit is conducted by a study coordinator. During this visit, the participant’s understanding of and ability to comply with the study requirements is confirmed and demographic information is obtained. Consenting participants are also invited to enroll in the online Fox Insight study at this time.

Prior to the baseline video visit, participants complete Parts IB and II of the Movement Disorder Society – Unified Parkinson Disease Rating Scale (MDS‐UPDRS) relating to motor and nonmotor features of PD, 26 a falls assessment, and patient global impression (PGI) of symptom severity in REDCap, as detailed in the schedule of activities (Table 1). At the baseline video visit, a study coordinator (EB, SD, TM) collects concomitant medications, administers the Montreal Cognitive Assessment (MoCA), 27 and enrolls and orients participants to the tasks in the mPower 2.0 smartphone app. Participants are mailed or emailed the portions of the MoCA that require a participant‐written response in advance of the visit and instructed not to complete the tasks prior to the video visit. Responses are scored by asking participants to display their completed items to their web camera. A movement‐disorders trained neurologist (JA, KA, RS, CT) then administers Parts IA, III, and IV of the MDS‐UPDRS, the Schwab & England Activities of Daily Living Scale (SEADL), 28 a falls assessment, and clinical global impression (CGI) of symptom severity. 29

Table 1.

AT‐HOME PD Assessment Measures

Video Visits

Previsit:

  • MDS‐UPDRS parts IB (Nonmotor Aspects of Experiences of Daily Living) and II (Motor Aspects of Experiences of Daily Living)

  • PGI of change and severity

  • Self‐reported falls

During Visit:

  • MDS‐UPDRS parts IA (Nonmotor Aspects of Experiences of Daily Living), III (Motor Examination) performed in the usual ON state, and IV (Motor Complications)

  • MoCA

  • SEADL

  • CGI of change and severity

  • Determination of falls assessment

Post Visit:

  • Preference and burden survey

mPower 2.0 Smartphone App

Quarterly Motor Tasks:

  • Finger tapping task ‐ 30 seconds of rapid finger tapping in each hand

  • Resting tremor task ‐ 30 seconds of holding the phone in each hand

  • Gait task ‐ 30 seconds of walking

  • Balance task ‐ 30 seconds of standing still

Continuous Passive Monitoring:

  • Location services on phone to capture movement patterns

  • Accelerometer gyroscope to capture walking patterns

  • Pedometer to capture the number of steps taken

Fox Insight

Quarterly Surveys:

  • Parkinson’s Disease Questionnaire‐8 (PDQ‐8)

  • Nonmotor Symptoms Questionnaire (NMS‐QUEST)

Bi‐Annual Surveys:

  • MDS‐UPDRS part II

  • Your Cognition and Daily Activities (PDAQ‐15)

  • Euroqol Five (EQ‐5D)

  • Parkinson’s Disease‐Patient Report of Problems (PD‐PROP)

Annual Surveys:

  • Physical Activity Scale for the Elderly (PASE)

  • Geriatric Depression Scale (GDS‐15)

Part III of the MDS‐UPDRS is modified to exclude assessments of rigidity and postural instability, which cannot be performed remotely. The validity of the remote assessment of Part III of the MDS‐UPDRS has been previously evaluated in two small studies, 30 including a sub‐study of STEADY‐PD III in which 38 participants underwent remote assessment within 4 weeks of in‐person assessment with moderate correlation in scores. 31 All study investigators are movement disorders‐trained neurologists who have completed MDS‐UPDRS certification. Additionally, before study initiation, the four investigators conducting the video visits independently rated five video‐recorded remotely performed MDS‐UPDRS part III motor examinations. The estimated intra‐class correlation for inter‐rater reliability was 0.65 and the videos were then reviewed as a group to improve inter‐rater reliability.

Similar outcome assessments are included in subsequent annual visits along with the addition of the clinician and patient global impression of change. 29 After each video visit, participants are surveyed on their experience (technical quality, comfort, and convenience) with the video visits based on surveys developed from previous studies. 32 , 33 Data from each video visit are entered directly into REDCap.

The study includes several features designed to maintain the safety of participants. Investigators and coordinators are provided with a workflow for how to manage medical issues that arise during a visit. Participant location is collected during the visit so that we may contact medical services in the event of a medical emergency. Additionally, precautions are taken to minimize the risk of falls during the performance of the MDS‐UDPRS motor examination. Specifically, participants are allowed to defer assessment of standing and gait and, when applicable, are asked to walk only with their assistive device.

PD smartphone application (mPower 2.0)

Consented participants enroll in the mPower Progression study, a remote study tracking the progression of PD using the mPower 2.0 smartphone application. The first version of the application was used by over 19,000 individuals with and without PD; the results of which have been previously published. 34 , 35 , 36 , 37 , 38 The application includes active tasks of motor function (finger tapping, rest tremor, gait and balance assessed using embedded sensors in the smartphone) and optional passive monitoring (displacement tracking to capture a participant’s lifespace 39 (i.e., their mobility patterns) and accelerometer and gyroscope readings to measure gait during daily living) (Table 1).

AT‐HOME PD uses a burst design in which participants are asked to complete assessments in mPower 2.0 for 14 days every quarter (four times annually) for the duration of the 24‐month study. During each 14‐day session, participants are asked to complete a full set of motor tasks on at least 10/14 days. Given the presence of diurnal fluctuation in task performance and changes in response to levodopa, 40 participants are asked to consistently complete the assessments either in the morning or the afternoon and to record the timing of their PD medication intake. To ensure that passive data collection is reflective of day‐to‐day life, participants are not given any instructions on when or how to carry their smartphone. Participants also complete a full set of the motor tasks during each video visit.

Fox Insight

Fox Insight is a prospective, online participant completed observational research study that aims to better understand experiences of daily living with PD. 41 Over 45,000 individuals with and without PD have already enrolled in this study and enrollment is ongoing. 42 AT‐HOME PD participants are asked to enroll in Fox Insight and complete web‐based surveys on a regular basis. The Nonmotor Symptoms Questionnaire and Parkinson’s Disease Questionnaire‐8 are completed quarterly. Part II of the MDS‐UPDRS, Physical Activity Scale for the Elderly, The Penn Parkinson’s Daily Activities Questionnaire‐15, Geriatric Depression Scale‐15, and PD‐Patient Report of Problems (which captures verbatim free‐text narratives about bothersome problems and their effect on daily functioning), 43 among others are completed on a less frequent basis (Table 1).

Data management

Data collected through video visits, mPower, and Fox Insight are deposited and aggregated in Synapse, which is a Sage Bionetworks‐maintained cloud‐based data management and research collaboration platform (https://www.synapse.org/). AT‐HOME PD data will be transferred to the Parkinson’s Disease Biomarkers Program (PDBP) Data Management Resource (DMR) (https://pdbp.ninds.nih.gov/) on an on‐going basis and ultimately merged with data from the parent clinical trials of isradipine and inosine. These studies of isradipine and inosine included deep phenotypic characterization, collection of DNA (for now completed whole‐genome sequencing [https://amp‐pd.org/whole‐genome‐data]) and other biological markers, and dopamine transporter imaging in a subset of participants, all of which will become part of NINDS’ PDBP.

Sub‐study

An initial objective was to evaluate the accuracy of the calibrated remote MDS‐UPDRS motor scores against in‐person MDS‐UPDRS motor scores collected during the last SURE‐PD3 study visit (ideally conducted within 28 days of the video visit). However, due to the early closure of SURE‐PD3, only 24 AT‐HOME PD baseline visits occurred within 28 days of the last SURE‐PD3 study visit. In order to meet this objective and also validate the remote assessment of the MoCA, we have initiated a sub‐study. For this sub‐study, we aim to recruit 50 PD participants from an on‐going, NINDS‐funded, University of Rochester study (Sensor Use to monitor Progression and Evaluate Symptoms Remotely in Parkinson’s Disease [SUPER‐PD]) 44 that is examining four different technologies (including mPower 2.0) for the assessment of PD disability and progression. Sub‐study participants are asked to complete a single video visit within 14 days of an in‐person study visit and permit the sharing of data from their in‐person study visit with the AT‐HOME PD study team. During this video visit, a study coordinator administers the MoCA and an investigator administers the MDS‐UPDRS motor examination.

Statistical analysis

The statistical analysis for the study is focused on (1) establishing the concordance between in‐person and video‐based clinical assessments, (2) comparing video‐based assessments and smartphone‐based assessments including the ability to distinguish subgroups with differing prognoses and predict key clinical events, and (3) determining feasibility.

Analysis will include the use of Lin's concordance correlation coefficient, ρc, 45 to evaluate the accuracy of the calibrated video visit estimates against assessments collected from the last in‐person study visit (either in SURE‐PD3 or SUPER‐PD). The accuracy of the remote assessment will be accepted as sufficient if ρc> 0.8. If we observe insufficient concordance, we will investigate the contributions of bias, scale difference, and imprecision.

Additional analyses include the assessment of the calibration and correlation between the video visits and smartphone assessments both cross‐sectionally and longitudinally. Analyses will generate smartphone‐derived metrics that best match composite scores of motor (e.g., MDS‐UPDRS part III) and global function (e.g., CGI, PGI, and SEADL). These measures are derived from machine learning models trained on data from other studies where MDS‐UPDRS has been collected in the clinic and mPower assessments were completed in clinic and at home. The accuracy of these measures will be evaluated using Lin's concordance correlation coefficient, ρc. If insufficient concordance is observed, we will investigate contributions of bias, scale difference, and imprecision.

Direct comparison between video visit and smartphone‐based measures of disease progression generally, as predictors of future clinical events (e.g., dropping below 80% on the modified SEADL and experiencing a fall), and between groups of participants with different anticipated rates of progression will be performed. Completion rates of patient‐reported outcomes collected during video visits and those collected online through Fox Insight will be compared by mixed effect logistic regression with participant‐specific and participant by form random effects. Concordance between patient‐reported outcomes assessed during video visits versus those obtained online through Fox Insight will be estimated using Lin's concordance correlation coefficient.

Power calculations

With approximately 75 participants contributing data to the comparison of in‐person versus video‐based assessments and approximately 225 participants contributing data to the comparisons of video‐based versus smartphone assessments, the study will have 80% power to declare a given measure sufficiently accurate (ρ c> 0.80 at alpha = 0.05) if the true concordance is at least 0.88 or 0.85, respectively.

Results

The NIH issued the notice of award on June 26, 2018, and the study was approved by the University of Rochester’s institutional review board on September 18, 2018. From October 2, 2018, to December 6, 2019, 226 enrolled in the study from 42 United States and 1 Canadian province (Figures 2 and 3). Of these, 123 (54%) came from STEADY‐PD III and 103 (46%) from SURE‐PD3. We enrolled 36% of the initial 634 clinical trial participants and 45% of the potentially eligible 504 clinical trial participants, those who consented to be contacted for future research and whose participation was not prematurely concluded. Fifty‐four of the 56 STEADY‐PD III and 50 of the 54 SURE‐PD3 clinical trial sites are represented.

Figure 2.

Figure 2

CONSORT flow diagram of participants in AT‐HOME PD.

Figure 3.

Figure 3

Location of participants in AT‐HOME PD.

We were unable to contact 116/504 potentially eligible participants. Of those whom we were able to contact but did not enroll, 40 expressed initial interest but were unable to be reached for prescreening, nine were ineligible, and 99 declined participation. Of the nine ineligible individuals, three were deceased, two had dopamine transporter scans inconsistent with PD, two had no access to internet or devices necessary for participation, one did not speak English, and one had withdrawn from the parent study. The most frequent reasons for declining study participation (individuals could cite multiple) were (1) lack of interest in on‐going research participation (n = 57), (2) too busy (n = 25), (3) lack of comfort with the required technology (n = 18), (4) concerns regarding the study’s time requirements (n = 9), and (5) desire to participate in an interventional drug trial (n = 5).

Of the 226 who enrolled, 132 (59%) enrolled in both the mPower Progression and Fox Insight studies; an additional 49 (22%) enrolled in mPower only and 29 (13%) in Fox Insight only. Of the participants who enrolled in mPower, 46 used Android devices and 143 iOS devices with some users contributing data from multiple devices. 6066 smartphone‐completed motor assessments and 3,992 patient‐reported surveys were collected within 30 days of the baseline visit across 79% of enrollees. To date, 79 (43%) of smartphone participants have opted into passive data collection. Passive data collection did not become available until July 3, 2019, and is currently only available on iOS. As of July 7, 2020, 191 individuals have completed their month 12 visit, for a retention rate of 88%.

As of March 9, 2020, five individuals have enrolled in the sub‐study and completed the single required study visit. Enrollment in the sub‐study was temporarily halted as in‐person visits for the parent observational study were stopped due to the COVID‐19 pandemic but resumed as of July 2020.

Discussion

AT‐HOME PD evaluates the feasibility and utility of remote video visits and smartphone‐based assessments as digital measures of PD progression in a long‐term study. Pairing the phenotypic characterization and biological data obtained in the parent studies with long‐term remote digital assessments will add new dimensions to the characterization of disease progression.

AT‐HOME PD also addresses fundamental shortcomings in current clinical research for PD. First, the study brings research to participants instead of participants to research. Travel to research sites by individuals with PD may be limited by time, cost, work, and family commitments early in the disease, and is increasingly constrained by declining mobility, loss of driving ability, and/or growing caregiver burden as the disease progresses. 46 , 47 , 48 Most current studies only add to this burden, 49 particularly in trials of treatments to slow the progression of PD, which often require participants to travel to clinical sites every few months for years. Beyond disruptive inconvenience to participants, which can dissuade enrollment and distort outcomes, patients and clinicians are newly sensitized to health risks of nonessential, including research, travel by the COVID‐19 pandemic. 50 Although telephone calls have long been used as remote assessments in PD trials, these have typically been employed in between in‐person visits to monitor safety and medications rather than parkinsonian features and efficacy. Virtual visits, in contrast, enable the direct assessment of parkinsonian features and meaningful measurement of disease progression.

Second, the study enables the collection of frequent, objective data in real‐world home settings. Symptom fluctuations are inherent to PD, yet most studies rely on infrequent assessments at arbitrary times in artificial environments to assess the efficacy of interventions. Such a reliance, coupled with insensitive, subjective measures, increases the likelihood of missing a therapeutic benefit of a drug or device when one is present. Moreover, digital tools can continue to be used to collect data throughout major events, such as pandemics, that otherwise disrupt research and may offer novel and real‐time insights into the effect of such events on patients. Smartphone studies to date have focused on distinguishing healthy controls from PD participants 51 , 52 and establishing acceptable correlation between smartphone measures and traditional in‐clinic motor assessments. 53 , 54 , 55 However, validation against “gold standard” clinical outcome measures may not be the ideal approach if digital biomarkers are in fact better measures of disease progression, more sensitive to change, or capture novel features of disease progression or disability. 56 Few studies have examined a longitudinal change in smartphone measures (which are limited by a substantial drop‐off in participation over time). 57 By following participants longitudinally, data from AT‐HOME PD will allow us to objectively compare the accuracy of smartphone‐generated measures versus in‐person and video‐based assessments to detect a relevant functional change and to predict meaningful clinical events.

Consistent with previous virtual studies in PD, early results from the on‐going AT‐HOME PD study have demonstrated the feasibility of remote assessments, the potential for wide geographic participation from a central site, and a high level of satisfaction with this emerging research model. 10 , 11 Here, we added a virtual observational study to the end of a clinical trial. Such a research model could also be used, for example, to assess the long‐term safety and efficacy of pharmacological, behavioral or surgical therapies in real‐world settings. Additional approaches might use virtual studies to assess the natural history of a disease or a genetically defined sub‐population in advance of a clinical trial. Some interventional clinical trials have been conducted entirely remotely, a practice that has just begun in PD. 21 , 58 , 59

It is certainly plausible that virtual studies will become a framework for pragmatic long duration interventional or phase IV studies. Trials with lower‐risk interventions (e.g., exercise) and those with one‐time or infrequent interventions may be more amenable to a virtual framework. In the near term, a hybrid approach with a combination of in‐clinic visits (for biological sample collection), in‐home visits (for blood collection and assessment of vital signs), and virtual visits (for additional safety and efficacy assessments) seems most attainable. However, more work is needed to adopt virtual designs for interventional studies. Valid concerns exist regarding the interpretation of within‐subject changes in MDS‐UPDRS motor assessments when using a combination of modified virtual MDS‐UPDRS and in‐person MDS‐UPDRS motor assessments. 60 These concerns highlight the need for additional validation studies, such as this one, and for the development of an analytical approach that enables the interpretation of a mix of in‐person and virtual assessments.

This study had significant recruitment challenges and limitations resulting in lower than targeted enrollment. With a sample size of 226 participants, approximately 75 participants will contribute to validation tests and this will provide 80% power to declare a given smartphone measure sufficiently accurate if the true concordance is at least 0.85. Some of these challenges are related to the manner in which the study was implemented. The virtual study was added late in the conduct of both phase 3 clinical trials posing logistical and operational challenges for research sites and research participants. Participant disappointment over the negative primary results of both parent trials likely lessened enthusiasm for enrollment in an extension study. These challenges were likely exacerbated by the premature termination of the inosine clinical trial due to futility, accelerating the planned time frame for transitioning research participants into the virtual study. As a result, few participants completed video assessments in close temporal proximity to in‐person assessments and our ability to assess correlations between in‐person and video assessments was limited. To enable much needed validation of the MDS‐UPDRS motor examination and MoCA, we are enrolling a cohort of approximately 50 individuals from an on‐going observational study of digital tools for the assessment of PD who will complete a single video‐based visit.

Virtual studies have challenges of their own. In this study, we relied on study participants to provide their own smartphones, the hardware, and software of which are variable. Some other studies have shied away from the “bring your own device” approach, and for clinical trials where the device is likely to contribute important outcome measures, providing all participants with a device may be advantageous. Similarly, relying on individuals to have their own device may further encourage a digital divide—the differential access to technologies based on socio‐economic, geographic, broadband access, or other factors. 61 In this study, we offered to cover the cost of internet access and a data plan for racial and ethnic minorities and received a single request, which was ultimately withdrawn. Additionally, the study team provided a web camera to 14 individuals to facilitate study participation. Nonetheless, the digital divide is real, and while virtual studies may overcome geographic barriers to participation, social barriers may remain more challenging. Expansion of broadband internet access is critical and urgently needed.

In the setting of the ongoing COVID‐19 pandemic, more work is urgently needed to validate virtual assessments, develop meaningful digital measures, and expand access to research participation in PD and other diseases. We are committed to facilitating this work and have made our protocol and model consent form available to researchers (https://www.athomepd.org/professionals).

In conclusion, AT‐HOME PD represents advancement in the conduct of PD research studies. Despite challenges with implementation and recruitment, the study has enrolled over 200 participants throughout the United States and Canada from two recently completed clinical trials. The study will inform longitudinal follow‐up of these two clinical trial cohorts as well as the feasibility, validity and comparative value of remote assessments, digital measures, and patient‐reported outcomes in an entirely virtual study. Ideally, learnings from this study will enable broad and large‐scale research participation in studies and trials that are participant‐centric and accelerate the development of much needed novel therapies for a common and disabling condition.

Conflicts of interest

Ruth B Schneider, MD – Dr. Schneider reports grants from Acadia Pharmaceuticals, Biohaven Pharmaceuticals, The Michael J. Fox Foundation for Parkinson’s Research, NIH/NINDS, and CHDI outside the submitted work. Larsson Omberg, PhD – Dr. Omberg was an employee of Sage Bionetworks during the conduct of the study and report grants from The Michael J. Fox Foundation for Parkinson’s Research, Novartis Pharmaceuticals, Celgene, NIH/NIA, NIH/NIMH, NIH/NINDS, Alzheimer's Drug Discovery Foundation, The Gates Foundation, The Wellcome Trust, and The Leona M. and Harry B. Helmsley Charitable Trust outside the submitted work. Eric A Macklin, PhD – Dr. Macklin reports grants from NIH/NINDS during the conduct of the study; grants from Acorda Therapeutics, Amylyx Pharmaceuticals, GlaxoSmithKline, and Mitsubishi Tanabe Pharmaceuticals; and other from Biogen, Cerevance, Acorda Therapeutics, Novartis Pharmaceuticals, Shire Human Genetic Therapies, and Stoparkinson Healthcare Systems outside the submitted work. Margaret Daeschler, BA – Ms. Daeschler was an employee of the Michael J. Fox Foundation during the conduct of the study. Lauren Bataille, MS – Ms. Bataille has nothing to disclose. Shalini Anthwal, PhD, MSc – Dr. Anthwal reports grants from NINDS during the conduct of the study. Taylor L Myers, BA – Ms. Myers reports grants from NINDS during the conduct of the study. Elizabeth Baloga, MPH – M. Baloga reports grants from NIH/NINDS and nonfinancial support from the Michael J. Fox Foundation for Parkinson's Research during the conduct of the study; grants from the Michael J. Fox Foundation for Parkinson's Research, and nonfinancial support from Parkinson's Foundation outside the submitted work. Sidney Duquette, BS – Mr. Duquette has nothing to disclose. Phil Snyder, BS – Mr. Snyder has nothing to disclose. Katherine Amodeo, MD – Dr. Amodeo was supported by the Michael J Fox Edmund J. Safra Fellowship during 2017‐2019. Outside the submitted work, KA has served as co‐investigator or medical monitor for clinical trials supported by Genentech Roche Ltd, EIP Pharma Inc, The Michael J Fox foundation for PD research, NINDS, Acadia Pharmaceuticals Inc, and Biogene. Christopher G Tarolli, MD – Dr. Tarolli has nothing to disclose. Jamie L Adams, MD – Dr. Adams reports grants from Biogen, National Institutes of Health/National Institute of Neurological Disorders and Stroke, and The Michael J. Fox Foundation for Parkinson’s Research outside the submitted work. Katherine F Callahan, BS – Ms. Callahan reports grants from NIH during the conduct of the study. Joshua Gottesman, MA – Mr. Gottesman has nothing to disclose. Catherine M Kopil, PhD – Dr. Kopil reports sub‐award on NIH grant, and other from The Michael J. Fox Foundation for Parkinson's Research during the conduct of the study. Codrin Lungu, MD – Dr. Lungu has nothing to disclose. Alberto Ascherio, MD, DrPH – Dr. Ascherio has nothing to disclose. James C Beck, PhD – Dr. Beck has nothing to disclose. Kevin Biglan, MD, MPH – Dr. Biglan reports no disclosures other than Eli Lilly outside the submitted work. Alberto J Espay, MD, MSc ‐ Dr. Espay reports grants from NIH, Great Lakes Neurotechnologies, and The Michael J Fox Foundation, and personal fees from Abbvie, TEVA, Impax, Acadia, Acorda, Cynapsus/Sunovion, Lundbeck, US WorldMeds, UCB, Lippincott Williams & Wilkins, Cambridge University Press, and Springer outside the submitted work. Caroline Tanner, MD, PhD – Dr. Tanner reports grants from The Michael J Fox Foundation during the conduct of the study; grants from Parkinson Foundation, Gateway LLC, Roche/Genentech, The Michael J Fox Foundation, NIH/NIA, NIH/NINDS, VA Merit, The U.S. Department of Defense, Biogen Idec and Parkinson Study Group; personal fees from Biogen Idec, Acorda, Adamas Therapeutics, Amneal, CNS Ratings, Grey Matter LLC, Northwestern University, Partners, Guidemark Health, Acadia, Neurocrine, Lundbeck, Cadent, and Harvard University; and nonfinancial support from Medtronic, Inc., Acadia, Boston Scientific, Neurocrine, and Biogen Idec outside the submitted work. David Oakes, PhD – Dr. Oakes has nothing to disclose. Ira Shoulson, MD – Dr. Shoulson reports other from Grey Matter Technologies (GMT), Biogen, and Jazz Pharmaceuticals and grants from The Michael J Fox Foundation for Parkinson's Research outside the submitted work. Dan Novak, PhD – Dr. Novak has nothing to disclose. Elise Kayson, MS – Ms. Kayson has nothing to disclose. E. Ray Dorsey, MD ‐‐ Dr. Dorsey reports grants from NINDS during the conduct of the study; grants from Burroughs Wellcome Fund, Davis Phinney Foundation, Food and Drug Administration, Greater Rochester Health Foundation, Huntington Study Group, The Michael J. Fox Foundation, National Science Foundation, Patient‐Centered Outcomes Research Institute, Safra Foundation, grants from Duke University, University of California Irvine, grants from Abbvie, Acadia Pharmaceuticals, AMC Health, BioSensics, GlaxoSmithKline, Nuredis Pharmaceuticals, Pfizer, Prana Biotechnology, Raptor Pharmaceuticals, Roche, Teva Pharmaceuticals, personal fees from 23 and Me, Abbott, Abbvie, American Well, Biogen, BrainNeuroBio, Clintrex, Curasen Therapeutics, DeciBio, Denali Therapeutics, GlaxoSmithKline, Grand Rounds, Karger, Lundbeck, MC10, MedAvante, Medical‐legal services, Mednick Associates, National Institute of Neurological Disorders and Stroke, Olson Research Group, Optio, Origent Data Sciences, Inc., Otsuka, Prilenia, Putnam Associates, Roche, Sanofi, Shire, Spark, Sunovion Pharma, Teva, Theravance, UCB and Voyager Therapeutics, outside the submitted work; editorial services for Karger Publications; and ownership interests with Blackfynn and Grand Rounds. Lara Mangravite, PhD ‐‐ Dr. Mangravite reports grants from the National Institute of Health during the conduct of the study; grants from Novartis, Genentech/Roche, Celgene, and BMS outside the submitted work. Michael A Schwarzschild, MD, PhD – Dr. Schwarzschild reports grants from NIH during the conduct of the study. Tanya Simuni, MD – Dr. Simuni reports grants from NINDS, The Michael J Fox Foundation, Parkinson’s Foundation, Biogen, Roche, Neuroderm, Sanofi, Sun Pharma, Abbvie, IMPAX, and Prevail; other from Acadia, Abbvie, Accorda, Adamas, Allergan, Amneal, Aptinyx, Denali, General Electric (GE), Kyowa, Neuroderm, Neurocrine, Sanofi, Sinopia, Sunovion, Roche, Takeda, Voyager, and U.S. World Meds during the conduct of the study.

Author contributions

ERD, LM, MAS, TS, and LO serve as principal investigators of the study. RBS, KA, CGT, JLA, TLM, SD, SA, and EB executed the study. MD, LB, and JG oversaw the Fox Insight portion of the study. LO and PS oversaw the mPower portion of the study. LO, PS, and EAM conducted the data analyses. KK, AA, JB, KB, ERD, AJE, RH, EK, CT, EM, LO, DO, MAS, IS, TS, DN, and CL serve as the study’s steering committee, conceived of and designed the study, contributed to the development of the protocol, and provide continuing oversight for the study. RBS, ERD, EAM, and LO drafted the manuscript. All authors edited, critiqued, and approved the final manuscript.

Acknowledgments

Research reported in this publication was supported by the PDBP of the National Institute of Neurological Disorders and Stroke of the National Institutes of Health under award number U01NS107009. We would also like to acknowledge and thank the many AT‐HOME PD participants who made this research possible.

Clinicaltrials.gov: NCT03538262

Funding Information

Research reported in this publication was supported by the PDBP of the National Institute of Neurological Disorders and Stroke of the National Institutes of Health under award number U01NS107009.

Funding Statement

This work was funded by National Institute of Neurological Disorders and Stroke grant U01NS107009.

REFERENCES

  • 1. Marras C, Beck JC, Bower JH, et al. Prevalence of Parkinson's disease across North America. NPJ Parkinsons Dis. 2018;4:21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Tufts Center for the Study of Drug Development . CNS Drugs Take Longer to Develop and Have Lower Success Rates than Other. 2014.
  • 3. Shults CW, Oakes D, Kieburtz K, et al. Effects of coenzyme Q10 in early Parkinson disease: evidence of slowing of the functional decline. Arch Neurol. 2002;59:1541–1550. [DOI] [PubMed] [Google Scholar]
  • 4. Parkinson Study Group QEI , Beal MF, Oakes D, et al. A randomized clinical trial of high‐dosage coenzyme Q10 in early Parkinson disease: no evidence of benefit. JAMA Neurol. 2014;71:543–552. [DOI] [PubMed] [Google Scholar]
  • 5. Dorsey ER, Papapetropoulos S, Xiong M, Kieburtz K. The first frontier: digital biomarkers for neurodegenerative disorders. Digital Biomarkers. 2017;1:6–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Schneider MG, Swearingen CJ, Shulman LM, Ye J, Baumgarten M, Tilley BC. Minority enrollment in Parkinson's disease clinical trials. Parkinsonism Relat Disord. 2009;15:258–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Simuni T, Siderowf A, Lasch S, et al. Longitudinal change of clinical and biological measures in early Parkinson's Disease: Parkinson's progression markers initiative cohort. Mov Disord 2018;33:771–782. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Dorsey ER, Venuto C, Venkataraman V, Harris DA, Kieburtz K. Novel methods and technologies for 21st‐century clinical trials: a review. JAMA Neurol 2015;72:582–588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Tanner CM, Meng CC, Ravina B, et al. A practical approach to remote longitudinal follow‐up of Parkinson's disease: the FOUND study. Mov Disord 2014;29:743–749. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Dorsey ER, Wagner JD, Bull MT, et al. Feasibility of virtual research visits in fox trial finder. J Parkinsons Dis 2015;5:505–515. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Dorsey ER, Darwin KC, Mohammed S, et al. Virtual research visits and direct‐to‐consumer genetic testing in Parkinson's disease. Digit Health. 2015;1:2055207615592998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Espay AJ, Bonato P, Nahab FB, et al. Technology in Parkinson's disease: challenges and opportunities. Mov Disord 2016;31:1272–1282. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Maetzler W, Domingos J, Srulijes K, Ferreira JJ, Bloem BR. Quantitative wearable sensors for objective assessment of Parkinson's disease. Mov Disord 2013;28:1628–1637. [DOI] [PubMed] [Google Scholar]
  • 14. Kabelac Z, Tarolli CG, Snyder C, et al. Passive monitoring at home: a pilot study in Parkinson disease. Digital Biomarkers. 2019;3:22–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Lipsmeier F, Taylor KI, Kilchenmann T, et al. Evaluation of smartphone‐based testing to generate exploratory outcome measures in a phase 1 Parkinson's disease clinical trial. Mov Disord 2018;33:1287–1297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Administration USFD . Real‐World Evidence. 2019; https://www.fda.gov/science‐research/science‐and‐research‐special‐topics/real‐world‐evidence.
  • 17. Papa SM, Brundin P, Fung VSC, et al. Impact of the COVID‐19 pandemic on Parkinson's disease and movement disorders. Mov Disord 2020;35:711–715. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Guidance for Industry, Investigators, and Institutional Review Boards . In: Administration FaD, ed. Rockville, MD2020:6–9.
  • 19. Schneider RB, Myers TL, Rowbotham HM, et al. A virtual cohort study of individuals at genetic risk for Parkinson's disease: study protocol and design. J Parkinsons Dis 2020;10:1195–1207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Smolensky L, Amondikar N, Crawford K, et al. Fox insight collects online, longitudinal patient‐reported outcomes and genetic data on Parkinson's disease. Sci Data 2020;7:67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Trial of Parkinson's And Zoledronic Acid (TOPAZ) . ClinicalTrials.gov.
  • 22. Biglan KM, Oakes D, Lang AE, et al. A novel design of a Phase III trial of isradipine in early Parkinson disease (STEADY‐PD III). Ann Clin Transl Neurol 2017;4:360–368. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Study of Urate Elevation in Parkinson's Disease, Phase 3 (SURE‐PD3) . National Library of Medicine 2018. https://clinicaltrials.gov/ct2/show/NCT02642393.
  • 24. von Elm E, Altman DG, Egger M, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 2007;335:806–808. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Bionetworks S. mPower. https://parkinsonmpower.org/.
  • 26. Goetz CG, Tilley BC, Shaftman SR, et al. Movement disorder society‐sponsored revision of the unified Parkinson's disease rating scale (MDS‐UPDRS): scale presentation and clinimetric testing results. Mov Disord. 2008;23:2129–2170. [DOI] [PubMed] [Google Scholar]
  • 27. Nasreddine ZS, Phillips NA, Bedirian V, et al. The montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695–699. [DOI] [PubMed] [Google Scholar]
  • 28. Schwab RS, England AC. Projection Technique for Evaluating Surgery in Parkinson's Disease. In: Gillingham FJ and Donaldson IML, eds 1969:152–157.
  • 29. Guy W. ECDEU Assessment manual for psychopharmacology . Rockville, MD: US Department of Health, Education, and Welfare Public Health Service Alcohol, Drug Abuse, and Mental Health Administration 1976.
  • 30. Stillerova T, Liddle J, Gustafsson L, Lamont R, Silburn P. Remotely assessing symptoms of Parkinson's disease using videoconferencing: a feasibility study. Neurol Res Int 2016;2016:4802570. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Tarolli CGAK, Bull MT, Goldenthal S, O'Brien M, Simuni T, Zimmerman G, Biglan KM, Dorsey ER. Virtual research visits in individuals with Parkinson disease enrolled in a clinical trial: REACT‐PD Study. American Academy of Neurology Annual Meeting; 2020; Presented Virtually.
  • 32. Beck CA, Beran DB, Biglan KM, et al. National randomized controlled trial of virtual house calls for Parkinson disease. Neurology. 2017;89:1152–1161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Tarolli CG, Zimmerman GA, Goldenthal S, et al. Video research visits for atypical parkinsonian syndromes among Fox Trial Finder participants. Neurology: Clinical Practice. 2019: 10.1212/CPJ.0000000000000680. [DOI] [PMC free article] [PubMed]
  • 34. Bot BM, Suver C, Neto EC, et al. The mPower study, Parkinson disease mobile data collected using ResearchKit. Sci Data 2016;3:160011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Sieberts SK, Schaff J, Duda M, et al. Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge. bioRxiv 2020;01:904722. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Chaibub Neto E, Pratap A, Perumal TM, et al. Detecting the impact of subject characteristics on machine learning‐based diagnostic applications. NPJ Digit Med 2019;2:99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Chaibub Neto E, Perumal TM, Pratap A, Bot B, Mangravite L, Omberg L. On the analysis of personalized medication response and classification of case vs control patients in mobile health studies: the mPower case study. arXiv:1706.09574. 2017.
  • 38. Perumal TM, Tummalacherla M, Snyder P, et al. Remote Assessment, in Real‐World Setting, of Tremor Severity in Parkinson's Disease Patients Using Smartphone Inertial Sensors New York, New York: ACM Press, 2018. Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers ‐ UbiComp '18 [Google Scholar]
  • 39. Liddle J, Ireland D, McBride SJ, et al. Measuring the lifespace of people with Parkinson's disease using smartphones: proof of principle. JMIR Mhealth Uhealth 2014;2:e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Zhan A, Mohan S, Tarolli C, et al. Using smartphones and machine learning to quantify parkinson disease severity: the mobile parkinson disease score. JAMA Neurol 2018;75:876–880. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Chahine LM, Chin I, Caspell‐Garcia C, et al. Comparison of an Online‐Only Parkinson's disease research cohort to cohorts assessed in person. J Parkinsons Dis 2020;10:677–691. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. The Michael J Fox Foundation for Parkinson's Research . Fox Insight. https://foxinsight.michaeljfox.org/.
  • 43. Javidnia MAL, Hosamath A, Eberly S, Oakes D, Nguyen A, Shoulson I. Comparison of categorical‐ and verbatim‐reported postural instability symptoms among fox insight Parkinson research participants. 2020 MDS Virtual Congress 2020.
  • 44. Dorsey ER, Omberg L, Waddell E, et al. Deep phenotyping of Parkinson's disease. J Parkinsons Dis 2020;10:855–873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45:255–268. [PubMed] [Google Scholar]
  • 46. Uc EY, Rizzo M, Anderson SW, Sparks J, Rodnitzky RL, Dawson JD. Impaired visual search in drivers with Parkinson's disease. Ann Neurol 2006;60:407–413. [DOI] [PubMed] [Google Scholar]
  • 47. Heikkilä VM, Turkka J, Korpelainen J, Kallanranta T, Summala H. Decreased driving ability in people with Parkinson’s disease. J Neurol Neurosurg Psychiatry 1998;64:325–330. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Martinez‐Martin P, Forjaz MJ, Frades‐Payo B, et al. Caregiver burden in Parkinson's disease. Mov Disord 2007;22:924–931. [DOI] [PubMed] [Google Scholar]
  • 49. Identifier: NCT03100149, A Study to Evaluate the Efficacy of Prasinezumab (RO7046015/PRX002) in Participants With Early Parkinson's Disease (PASADENA) . Clinicaltrials.gov
  • 50. Padala PR, Jendro AM, Gauss CH, et al. Participant and caregiver perspectives on clinical research during COVID‐19 pandemic. J Am Geriatr Soc. 2020;68:E14–E18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. Arora S, Venkataraman V, Zhan A, et al. Detecting and monitoring the symptoms of Parkinson's disease using smartphones: A pilot study. Parkinsonism Relat Disord 2015;21:650–653. [DOI] [PubMed] [Google Scholar]
  • 52. Arora S, Baig F, Lo C, et al. Smartphone motor testing to distinguish idiopathic REM sleep behavior disorder, controls, and PD. Neurology. 2018;91:e1528–e1538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Kassavetis P, Saifee TA, Roussos G, et al. Developing a tool for remote digital assessment of Parkinson's Disease. Mov Disord Clin Pract 2016;3:59–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Lee W, Evans A, Williams DR. Validation of a smartphone application measuring motor function in Parkinson's disease. J Parkinsons Dis. 2016;6:371–382. [DOI] [PubMed] [Google Scholar]
  • 55. Lee CY, Kang SJ, Hong SK, Ma HI, Lee U, Kim YJ. A validation study of a smartphone‐based finger tapping application for quantitative assessment of bradykinesia in Parkinson's disease. PLoS One 2016;11:e0158852. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Chen‐Plotkin AS, Albin R, Alcalay R, et al. Finding useful biomarkers for Parkinson's disease. Sci Transl Med 2018;10(454). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Pratap A, Neto EC, Snyder P, et al. Indicators of retention in remote digital health studies: a cross‐study evaluation of 100,000 participants. NPJ Digit Med. 2020;3:21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58. Orri M, Lipset CH, Jacobs BP, Costello AJ, Cummings SR. Web‐based trial to evaluate the efficacy and safety of tolterodine ER 4 mg in participants with overactive bladder: REMOTE trial. Contemp Clin Trials. 2014;38:190–197. [DOI] [PubMed] [Google Scholar]
  • 59. Smalley E. Clinical trials go virtual, big pharma dives in. Nature Biotechnol 2018;36:561. [DOI] [PubMed] [Google Scholar]
  • 60. Goetz CG, Stebbins GT, Luo S. Movement disorder society‐unified parkinson's disease rating scale use in the COVID‐19 era. Mov Disord 2020;35:911. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Norris P. Digital divide: Civic engagement, information poverty, and the Internet worldwide. Cambridge University Press, 2001.

Articles from Annals of Clinical and Translational Neurology are provided here courtesy of Wiley

RESOURCES