High-Resolution Tracking of Motor Disorders in Parkinson’s Disease During Unconstrained Activity

Abstract

Parkinson’s disease (PD) can present with a variety of motor disorders that fluctuate throughout the day, making assessment a challenging task. Paper-based measurement tools can be burdensome to the patient and clinician and lack the temporal resolution needed to accurately and objectively track changes in motor symptom severity throughout the day. Wearable sensor-based systems that continuously monitor PD motor disorders may help to solve this problem, although critical shortcomings persist in identifying multiple disorders at high temporal resolution during unconstrained activity. The purpose of this study was to advance the current state of the art by (1) introducing hybrid sensor technology to concurrently acquire surface electromyographic (sEMG) and accelerometer data during unconstrained activity and (2) analyzing the data using dynamic neural network algorithms to capture the evolving temporal characteristics of the sensor data and improve motor disorder recognition of tremor and dyskinesia. Algorithms were trained (n = 11 patients) and tested (n = 8 patients; n = 4 controls) to recognize tremor and dyskinesia at 1-second resolution based on sensor data features and expert annotation of video recording during 4-hour monitoring periods of unconstrained daily activity. The algorithms were able to make accurate distinctions between tremor, dyskinesia, and normal movement despite the presence of diverse voluntary activity. Motor disorder severity classifications averaged 94.9% sensitivity and 97.1% specificity based on 1 sensor per symptomatic limb. These initial findings indicate that new sensor technology and software algorithms can be effective in enhancing wearable sensor-based system performance for monitoring PD motor disorders during unconstrained activities.

Attempts at developing a wearable device that can automatically track changes in the presence and severity of involuntary motor disorders have focused primarily on Parkinson’s disease (PD). In addition to being among the most common neurodegenerative diseases among adults,¹ PD can present with a variety of different motor disorders that fluctuate throughout the day. Effective therapeutic management of these disorders depends on the ability of the clinician to accurately track their progression over time and in different parts of the body. The current means of tracking longitudinal changes in the patient’s motor status outside the clinic is dependent on the patient making entries into a motor diary. Diaries are prone to subjective errors and poor sensitivity when detecting change^4–7 and may be burdensome for individuals with PD, who are at risk for cognitive decline and dementia^2,3

Wearable, sensor-based devices for monitoring PD motor disorders are designed to record, analyze, and automatically interpret mechanical and/or physiological signals resulting from the patient’s voluntary and involuntary muscle activity. Recent advances in wearable sensor technology⁸ and improvements in machine learning algorithms⁹ have brought us closer to overcoming the inherent challenges of implementing such devices. Despite this prospect, no system is currently available that can remotely monitor PD motor disorders during unrestricted daily activities with sufficient temporal or spatial resolution to track the full complement of PD motor disorders and their fluctuations throughout the day.

The most common approaches to developing a PD monitor have relied on accelerometers^10–15 (ACCs), gyroscopes,^16–18 inertial sensors,¹⁹ and sEMG sensors.^20,21 Many of these devices have been validated to work reasonably well at identifying a single motor disorder such as resting tremor^18,20 or dyskinesia^12–15 during scripted activities. These restrictions simplify the task of identifying a disorder because confounding signals generated by normal extemporaneous daily activities are minimized. Other recent developments have focused on automating the administration of standardized motor assessment scales for PD disorders.^18,19,22,23 These approaches were designed primarily for identifying tremor and/or bradykinesia, and have not included other motor signs of PD or dyskinesia. They also shift the burden of timely administration from the clinician to the patient, which may be challenging because of the cognitive deficits that are characteristic of advanced Parkinson’s disease.³

This report describes sensor and data-processing technologies that achieve high temporal and spatial resolution for identifying the severity of tremor and dyskinesia using a minimal number of sensors during unconstrained activities.

Patients and Methods

Subjects

Two groups of subjects were tested (Table 1): 1 group (n = 11 with PD) provided a data set for algorithm development (training set), and the other group (n = 8 with PD; n = 4 without PD) provided data for testing the algorithms (test set). The acquisition of separate databases was implemented to demonstrate that the algorithms are subject-independent and need not require pretraining for each application. Patients were screened for mild to moderately severe categories of Parkinson’s disease (Hoehn–Yahr stages II–III while “on” and Hoehn–Yahr stages III–IV while “off”),²⁴ with a mean disease duration of 13 years for both groups. All were taking levodopa as well as other antiparkinsonian medications. The patients presented with tremor scores ranging from 0 to 4, based on the Unified Parkinson’s Disease Rating Scale (UPDRS)²³ and dyskinesia scores ranging from 0 to 4 based on the modified Abnormal Involuntary Movement Scale (m-AIMS).²⁵ None were diagnosed with dementia, and all were ambulatory. Non-PD subjects were selected to be within the age range of the patients and were screened for neuromuscular disorders, including PD. All subjects provided voluntary written informed consent approved by the Boston University institutional review board prior to their participation in the study.

TABLE 1.

Characteristics of the subject populations used for training and testing the dynamic neural network (DNN) algorithms

	Training set	Test set
PD patients
Number	n = 11	n = 8
Age (y)	61.1 ± 5.5	62.9 ± 5.3
Men/women	9/2	7/1
Disease duration (y)	13.5 ± 6.0	13.2 ± 9.2
Levodopa dose (mg/day)	1072.2 ± 788	930 ± 839
UPDRS (Motor Score)	37.6 (11.2)	39.5 (10.6)
Tremor prevalence (%)a	17.7 ± 19.3	16.7 ± 20.7
Mild/moderate/severe (%)c	—	54/34/12
Tremor duration (s)b	39.4 ± 43.0	42.3 ± 34.4
Dyskinesia prevalence (%)a	49.7 ± 45.1	48.5 ± 25.7
Mild/moderate/severe (%)c	—	47/41/12
Dyskinesia duration (s)b	52.3 ± 51.7	62.6 ± 45.6
Prevalence at rest (%)d	4.8 ± 2.2	4.6 ± 3.1
Subjects without PD
Number	n = 0	n = 4
Age (y)	—	54 ± 16.6
Men/women	—	4/0

^aPercentage of recording period, regardless of severity or body location.

^bBased on how long a disorder persisted at a particular severity level.

^cPercentage of total movement disorder duration in the severity categories.

^dPercentage of total recording period in which the subject displayed no voluntary activity.

Methods

Data Acquisition

Our goal is to develop a system that requires only 1 sensor per symptomatic limb for identifying tremor and dyskinesia in that limb. Accordingly, only 1 sensor was placed on each of the 4 extremities. Sensors were on the middle of the muscle belly (away from tendon and innervation zones) of the extensor carpi ulnaris (ECU) muscle in the upper limbs and of the tibialis anterior (TA) muscle in the lower limbs. Each hybrid sensor (Fig. 1) is instrumented with a triaxial accelerometer (dynamic range, ±6 g; maximum resolution, 0.0008 g/bit; bandwidth range, DC to 50 Hz) and sEMG sensing (gain of 1000; bandwidth range, 20–450 Hz; baseline noise, <1.25 μV root mean square [RMS]). The selection of the TA and ECU muscles was based on pilot experiments that indicated that these muscles are most active when tremor and dyskinesia are present in limb muscles. A reference electrode for the sEMG recordings was attached to the skin at the C7 bony prominence. Sensors were connected to a hip-worn data acquisition unit, and analysis was conducted offline on a PC workstation (sampling rate of 1000 Hz using a 16-bit A/D card).

FIG. 1.

Block diagram of the procedures used to detect and analyze surface electromyographic (sEMG) and accelerometer (ACC) signals from hybrid sensors. The sensor is configured with parallel sEMG detection bars on the bottom of the sensor and a triaxial accelerometer to provide X, Y, Z outputs to the data acquisition system. sEMG and ACC features are extracted from these data to serve as inputs to a dynamic neural network (DNN) for tremor (7 input nodes for each feature, 4 hidden nodes, and 1 output node). Features extracted from the ACC signal serve as inputs to the DNN for dyskinesia (4 input nodes for each feature, 2 hidden nodes, and 1 output node). Severity of each disorder is identified through the use of a maximum a posteriori probability (MAP) classifier.

Experimental sessions were continuously videotaped using fixed and handheld high-resolution digital cameras. Data were recorded continuously for approximately 4 hours (to capture a complete “on-off” medication cycle) in a 100-m² laboratory arranged to simulate a home environment. Video and sensor data were synchronized by generating a pulse tone recorded on the cameras audio channel. Sessions were timed to begin approximately 1 hour following the patient’s first morning dose of antiparkinsonian medication. The subjects were free to move about the simulated home environment without coaching from the researchers or use of an activity script. The physical and social environment was designed to favor voluntary activities that included a variety of mobility states (sitting, standing, walking, and lying down), during which numerous diverse activities occurred, such as preparing snacks, eating, reading, writing, and interacting with researchers and family members.

Data Analysis

Video Annotation

Video annotation for scoring tremor and dyskinesia severity was carried out by a team of movement disorder specialists consisting of 2 neurologists and a nurse coordinator from the Boston University Parkinson’s Disease Center and a physical therapist. This information provided the basis on which the classification algorithms were trained and tested. Tremor severity was scored based on item 20 (tremor at rest) and item 21 (action or postural tremor) of the Motor Examination section of the UPDRS.²³ Dyskinesia severity was scored based on the m-AIMS scale. Both instruments use a 5-point Likert scale, with 0 = the absence of the disorder and 4 = the most extreme disorder. Annotators identified the beginning and end of each movement disorder severity occurrence with a resolution of 1 second. Each of the 4 limbs was scored separately. Annotated scores of tremor and dyskinesia severity in the lower limb during walking could not be reliably observed by the movement disorder experts because of the speed of lower limb movement during gait. The algorithmic identification of these disorders during walking was therefore based on sensor data from the upper extremity.

Signal Processing and Analysis

The sEMG and ACC signals were analyzed to extract features in the time and frequency domain using RMS and autocorrelation-based parameters (Fig. 1) derived from previous data-mining studies that differentiated voluntary from involuntary movements.^26–29 These features were used as inputs to time-dependent dynamic neural networks (DNNs) that were implemented separately for tremor and dyskinesia using a multilayered feed-forward architecture. We implemented DNNs instead of the more traditional static neural networks¹⁴ because they are capable of learning time-dependent relationships between the inputs. Training was implemented using a temporal back-propagation algorithm.³⁰ The input features to the DNNs were calculated over a 2-second window, and the output produced a single value ranging between −1 (no disorder detected) and +1 (disorder detected) at the rate of 1 per second. The hidden nodes and the output node used the weights of a 5-point FIR filter applied to time-delayed and time-advanced versions of their respective input data.

When the tremor and dyskinesia motor disorders were identified, a simple Bayesian maximum a posteriori probability (MAP) classifier was applied to determine severity level, based on calculations of accelerometer energy.^31,32 The severity detector classified each second in which the disorder was present using 3 categories: “mild,” “moderate,” or “severe,” corresponding to a UPDRS and/or m-AIMS score of 1, 2, or 3–4, respectively. Scores of 3 and 4 were combined because of the relatively few scores of 4 (<10% of the disorder duration).

Evaluating the Classification Algorithms

Discrepancies between the classification algorithm and expert annotation were evaluated on the basis of sensitivity and specificity measurements. Sensitivity describes the ability of the algorithm to correctly identify a movement disorder when it is present, and specificity describes the ability of the algorithm to correctly identify all instances when the movement disorder is absent.^26–29

Based on these calculations, we computed the global error rate (GER) from a normalized set of testing data in which the number of seconds during which the disorder was present is equal to the number of seconds when the disorder was absent. The formula used for calculating the global error rate is: 1 − ([sensitivity + specificity]/2).

The GER was normalized to adjust for the possible influence of mobility state (ie, duration of sitting, standing, and walking) by calculating a separate GER for each of the 3 mobility states and calculating an average value. Without this normalization, algorithm performance could be exaggerated (eg, if the data being analyzed were primarily from quiet sitting). To measure the ability of the algorithm to avoid localized errors, we also determined a local error rate,^27,29 defined as the proportion of 30-second intervals in which more than half the classifications were incorrect.

Results

Voluntary Versus Involuntary Activity

The activity summary in Table 1 specifies that the monitoring periods contained a relatively high presence of both voluntary and involuntary motor activity, with a minimum of “at rest” states, thereby providing numerous instances in which involuntary movement disorders were differentiated from purposeful movements by the algorithms.

Signal Characteristics

Figure 2 highlights the differences in the signal characteristics for sEMG and accelerometer recordings of tremor and dyskinesia. Tremor produces periodic sEMG and accelerometer signal “bursts” that are relatively constant in duration and amplitude (Fig. 2A). In contrast, dyskinesia produces large irregular fluctuations in both the sEMG and accelerometer signals (Fig. 2C). These defining signal characteristics are not always so easily differentiated from normal voluntary movements, as exemplified in Figure 2D.

FIG. 2.

Raw surface electromyographic (sEMG) and accelerometer signal patterns characteristic of tremor (A), subclinical tremor (B), dyskinesia (C), and normal voluntary movement (D) are shown for data acquired from the wrist extensor location of a patient with PD. Tremor is characterized by repetitive sEMG signal bursts at a fixed frequency related to cyclic movements of the limb. The periodic sEMG activity is preserved during subclinical tremor but produces no observable limb movement. In contrast, dyskinesia is characterized by irregular sEMG activity and rapid chaotic movements. Normal voluntary activities such as feeding oneself (D) may produce rapid accelerations and bursts of muscle activation that can mimic dyskinesia (C), making accurate recognition challenging for the algorithms.

Algorithm Training

Algorithms were initially trained using 44 hours of patient data (n = 11 PD patients). Dyskinesia and tremor training and testing sets were segregated by limb. We were able to reduce the size of the training database set to 20 hours by iteratively removing redundancies in the type of physical activities recorded for a particular subject and retaining regions that provided a sufficient range of movement disorder severities occurring during the different unscripted activities. Training of the algorithms using the reduced data set achieved 100% sensitivity and 99% specificity for tremor and 98% sensitivity and 99% specificity for dyskinesia.

Algorithm Testing

The algorithms were tested on an independent data set consisting of 29 hours of data from a different group of 8 PD patients and 15 hours of data from 4 non-PD subjects (Table 1).

Tremor Results

The results are summarized in Table 2 for different severity levels, mobility states, and extremities. The algorithms were able to detect the occurrence of tremor with an overall mean sensitivity and specificity of 90.2% and 92.9%, respectively. The relatively low percentage of localized errors for the entire data set (mean, 1.9%) indicated that tremor detection was accomplished with relatively few localized clusters of errors. Tremor was best differentiated by the sEMG energy feature and by the ACC autocorrelation feature for quantifying the frequency content of the signal. The identification of mild, moderate, and severe subcategories was achieved with error rates less than 5% (Table 2).

TABLE 2.

Summary of algorithm performance

	Sensitivity	Specificity	Global error	Local error
Severitya
Tremor
Mild	97.2%	97.6%	2.7%	1.4%
Moderate	95.2%	97.1%	3.9%	1.8%
Severe	96.3%	99.3%	2.2%	1.7%
Dyskinesia
Mild	93.9%	95.5%	5.3%	7.1%
Moderate	91.9%	94.6%	6.8%	4.9%
Severe	95.0%	98.6%	3.2%	4.1%
Overall mean (SD)	94.9%	97.1%	4.0%	3.5%
	(1.9%)	(1.8%)	(1.7%)	(2.3%)
Mobility
Tremor (UE)
Sitting	95.1%	94.9%	4.9%	0.23%
Standing	88.5%	93.5%	8.9%	0.19%
Walking	96.5%	93.7%	4.9%	0.18%
Tremor (LE)b
Sitting	95.4%	89.6%	7.5%	3.20%
Standing	75.4%	95.8%	14.4%	4.95%
Overall mean (SD)	90.2%	93.5%	8.1%	1.75%
	(8.8%)	(2.4%)	(4.0%)	(2.2%)
Dyskinesia (UE)
Sitting	89.5%	97.7%	6.4%	1.6%
Standing	92.0%	94.6%	6.7%	3.9%
Walking	99.3%	75.7%	12.5%	10.9%
Dyskinesia (LE)b
Sitting	85.3%	96.0%	9.3%	2.0%
Standing	92.3%	83.5%	12.1%	2.4%
Overall mean (SD)	91.7%	89.5%	9.4%	4.2%
	(5.1%)	(9.5%)	(2.9%)	(3.9%)

Results are from a single sensor located on the symptomatic limb.

^aSeverity results are for upper and lower extremities across all mobility states. Local error was calculated based on local error rate > 50%, for consecutive 30-second intervals.

^bTremor and dyskinesia in the lower extremity during walking was not observable. UE, upper extremity; LE, lower extremity.

False-positive identification of tremor was produced in 3 of the patients tested during brief episodes of repetitive MU discharges at frequencies consistent with tremor, but without visible signs of tremor movement (Fig. 2B). These instances only occurred just prior to overt clinical signs of tremor.

The training and test databases for tremor were reanalyzed without the sEMG features to assess whether hybrid sensor data improves tremor recognition. The inclusion of sEMG features with ACC features resulted in a 10% improvement in overall sensitivity, a 33% improvement in the GER, and a 77% improvement in the LER compared with an ACC-only database. Specificity was unchanged, at approximately 94% for both conditions.

Dyskinesia Results

Dyskinesia recognition by the algorithms (Table 2) resulted in overall sensitivity and specificity of approximately 91.7% and 89.5%, respectively, which is comparable to the results we derived for tremor recognition. The algorithms were particularly effective in providing minimal local error rates in the arms (mean, 0.20%) compared with the legs (mean, 3.6%). The relatively low percentage of localized errors for the entire data set (mean, 1.9%) indicated that the errors were not concentrated in specific intervals but were generally more evenly distributed.

Rapid normal movements were distinguished from dyskinetic movements primarily from ACC amplitude and frequency parameters. The identification of mild, moderate, and severe subcategories of dyskinesia was achieved with errors that were similar to those reported for tremor.

Dose-Related Results

Figure 3 provides an example of the ability of the algorithms to accurately capture continuous dose–response movement disorder information from the upper extremity of a PD patient. The figure shows the rapid fluctuations in movement disorder severity and gradual transition from dyskinesia to tremor approximately 150 minutes following the patient’s first dose of anti-PD medication. Sensitivity ranged from 93.3% to 97.4% for the different severities of dyskinesia and from 94.5% to 97.2% for the different severities of tremor from these data. The lowest specificity was 97% for both disorders.

FIG. 3.

Plot of dyskinesia and tremor severity as a function of time following administration of the first dose of anti-Parkinson’s medication (l-dopa). Transitions between different movement disorders and different severities are plotted using a 1-second scale. For every second of data, there is only 1 movement disorder severity score. Black-diamond data points represent results from the algorithm, and gray-diamond data points represent results from the video annotations. The results were analyzed from a sensor on the right forearm of a 62-year-old subject with a 16-year history of PD complicated by motor fluctuations.

Discussion

This study introduces a new approach of combining hybrid sEMG and accelerometer sensor data with DNN analysis to provide accurate automatic detection of tremor and dyskinesia severity at a high temporal resolution in PD patients during unconstrained daily activities. This capability was achieved based on single-sensor data from symptomatic upper or lower extremities. Although other reports in the literature have achieved sensitivity and specificity results for tremor and dyskinesia recognition of approximately 90% or better, as in our study, they did so under constraints that we did not impose, such as scripted-activity monitoring conditions,^{14–16,18,19,22} low temporal resolution,^14,15,18 or the need for multiple sensors to identify a motor disorder in a symptomatic limb.^14,15,22 Our performance metrics were also achieved on the basis of the analysis of sensor data that were independent of the training data, thereby facilitating clinical ease of use in future applications. Movement disorder recognition algorithms designed to produce data points with a resolution measured in minutes or hours would have failed to capture the characteristically unpredictable, rapid, and transient nature of the motor disorder fluctuations in this PD population.

A goal of our study was to produce accurate motor disorder classifications using a software algorithm requiring a minimal number of miniaturized sensors. An important design consideration was to select sensor locations that were amenable to self-application and use under clothing. This goal was achieved by developing an adaptable 4-sensor approach to derive data from distal limb segments for ease of use. In its maximal configuration of 1 sensor per limb, a comprehensive assessment of the body can be achieved, as for instance when doing a baseline screening. For those patients who may have difficulty managing multiple sensors in their home, the number of sensors can be restricted to the most symptomatic or functionally important limb(s). Current wearable sensor solutions can require as many as 6¹⁵or 8²² sensors and are neither adaptable nor limb specific.

The use of a hybrid sensor in the current study originated from our previous work, in which we demonstrated that this combination provided advantages in training an artificial neural network to identify a variety of different activities of daily living when compared with an accelerometer-only approach.³³ The current study also compared recognition performance of the algorithms for inputs with and without combined sEMG and ACC data, documenting the value-added benefit of having both sets of data for tremor recognition. Further studies are needed to determine whether the identification of “subclinical” tremor using both sEMG and ACC signals can be considered a clinical advantage, for example, for early detection of PD in patients who are otherwise asymptomatic.

The analytic approach of processing sensor signals by DNNs to classify PD motor disorders is unique, although DNNs have been used effectively in neural prosthetics³⁴ and motor control studies.³⁵ Another unique aspect of our approach was to configure the DNNs so that each limb was separately assigned a classifier that operated independently from the other classifiers. The benefits of this approach were most apparent in our ability to track movement disorders during unconstrained daily functional activities. Despite the overall success of achieving this goal, the results must be considered an initial finding until further development and testing on a larger patient population are undertaken. We are actively investigating a new procedure that will integrate the DNNs within a larger artificial intelligence framework of our design, referred to as integrated processing and understanding of signals (IPUS),³⁶ to provide a rule-based structure that adaptively selects appropriate classifiers to further resolve recognition challenges.^29,37 Preliminary results for identification of freezing²⁹ and other gait abnormalities in PD²⁷ using this procedure have been encouraging.

Conclusions

This initial study demonstrated that a combined approach of hybrid sensor data with dynamic neural network processing achieved high temporal resolution for identifying limb-specific changes in the severity of tremor and dyskinesia during normal daily activities in patients with PD. The value-added benefits of including both sEMG and accelerometer data for identifying tremor were described. The incorporation of DNN analysis for capturing time-dependent changes enabled us to achieve disorder recognition accuracy during unconstrained daily activities comparable to that achieved by others for less challenging standardized activities.