The Manumeter: A non-obtrusive wearable device for monitoring spontaneous use of the wrist and fingers

Abstract

This paper describes the design and pilot testing of a novel device for unobtrusive monitoring of wrist and hand movement through a sensorized watch and a magnetic ring system called the manumeter. The device senses the magnetic field of the ring through two triaxial magnetometers and records the data to onboard memory which can be analyzed later by connecting the watch unit to a computer. Wrist and finger joint angles are estimated using a radial basis function network. We compared joint angle estimates collected using the manumeter to direct measurements taken using a passive exoskeleton and found that after a 60 minute trial, 95% of the radial/ulnar deviation, wrist flexion/extension and finger flexion/extension estimates were within 2.4, 5.8, and 4.7 degrees of their actual values respectively. The device measured angular distance traveled for these three joints within 10.4%, 4.5%, and 14.3 % of their actual values. The manumeter has potential to improve monitoring of real world use of the hand after stroke and in other applications.

I. Introduction

An important goal of stroke motor rehabilitation is to improve patients’ ability to use their upper extremity to perform needed activities of daily living. However, measuring individuals’ ability to use their impaired limb in the real world can be challenging. Measurements of upper extremity motor function performed in the clinic or laboratory may not accurately reflect actual use of the limb [1], [2]. Two standard tools for addressing this problem are the Motor Activity Log (MAL) and accelerometry, which have been used to estimate spontaneous use of the upper extremity in the real world [3–6]. The MAL involves an interview in which subjects are asked to report how often (amount) and how well (quality of use) they believe that they use their impaired limb for a set of common daily tasks [7], [8]. Although the MAL is attractive because of its simplicity, it is subjective because it is self-reported. The MAL also relies on the memory and comprehension of the subject, making it difficult for some patients [9]. Like other self-reported measures of activity, the MAL is only moderately correlated with direct measures of hand use such as accelerometry [8], [9].

Accelerometry can be a more objective means of measuring spontaneous use of the impaired hand [10], [11]. A common way to use accelerometry to quantify use of the upper extremity is to require subjects to wear a data-logging accelerometer enclosed in a watch-like unit. Accelerations recorded by these units are typically integrated over short time intervals to create an arbitrary unit called a raw count [10]. Although it has been demonstrated that the raw counts produced by an accelerometer worn at the wrist correlate well with movement speed and duration when averaged across subjects [12], [13], the measure is too noisy to provide reliable data on an individual basis [13], [14]. This noise can be reduced by using a threshold filter to increment a movement score only if accelerations measured at the wrist exceed a predefined threshold within a given time epoch [10], [13]. Using the threshold filter approach, such scores have been shown to correlate well with the total time spent moving the arm in daily living and to have good test-retest reliability for various upper extremity movement tasks [13].

Unfortunately, the noise reduction gained by using the threshold filtering of acceleration reduces sensitivity to movement quality and features. Small movements can be overlooked, and any combination of movements large enough to push the accelerations over the threshold will result in the same score for an epoch [13]. Moreover, because the sensors are worn at the wrist, accelerometry is insensitive to fine movements of the wrist and hand, such as writing or typing [13]. Given the importance of the wrist, hand, and fingers in many activities of daily living, it would be desirable to estimate their actual use.

In the laboratory, datagloves, goniometers, and motion capture systems can been used to quantify use of the wrist and hand [15–23]. However, such devices are typically not designed for long-term data-logging in an uncontrolled environment. They can be difficult for individuals with a physical impairment to don and doff, may restrict natural movement of the hand, and may be too cumbersome to wear for long periods of time. Such issues can be reduced through appropriate engineering (e.g. [18]), but still there is no device to our knowledge for measuring wrist, hand, and finger movement that is as unobtrusive as a normal garment or piece of jewelry .

To address these limitations, we have developed a novel device for unobtrusive monitoring of the wrist, hand, fingers called the manumeter – a term inspired by the term “pedometer”, which describes a non-obtrusive device that measures the distance traveled on foot as determined by the number of steps taken. The manumeter uses magnetometers mounted in a wristwatch-like unit and a magnetic ring worn on the finger to track the angular distance traveled by the wrist and fingers while it is worn.

II. Methods

A. Electrical design

The manumeter system consists of a sensing and data-logging unit (Fig. 2) located in a watch-like enclosure worn on the wrist and a small neodymium ring worn on the index finger (Fig. 1). Two triaxial magnetometers (Analog Devices HMC5943) located on the proximal and distal sides of the data-logger board measure the strength and orientation of the magnetic field produced by the ring (Fig. 2a.). A microcontroller (Microcontroller PIC24FJ64GB002) samples data from the magnetometers and a triaxial accelerometer (Analog Devices ADXL335), buffers it, and intermittently writes it to an on-board 16GB SD card (Fig. 2b). The unit draws approximately 20mA at 3.3V from a 3.7V, 450mAh LiPO battery (Fig. 2c.) that can power the device for over 20 hours.

Fig. 2.

Sensing and data-logging board used to collect and store manumeter data. On the top side of the PC board (1a), magnetometers on either end of the board measure movement of the magnetic ring, and an accelerometer measures accelerations of the arm. A microcontroller reads incoming sensor signals and writes them to a microSD card located on the back of the printed circuit board (1b). A LiPO battery (1c) powers the unit for more than 20 hours and is recharged via a USB cable when attached to a computer.

Fig. 1.

The manumeter consists of a sensing and data-logging board worn in a watch-like enclosure and a magnetic ring worn on the finger. The device senses wrist flexion/extension, wrist deviation, finger flexion/extension, and gross arm movement

The manumeter is connected to a computer to transfer the data collected by the microcontroller and to recharge the LiPO battery through a USB 2.0 A Male to Micro B cable. When connected to a host computer, the manumeter appears as a mass storage device, and a Java program running on the computer moves the files from the manumeter to its hard drive and then processes the data to assess use of the wrist and hand. The processed data can then be stored and/or presented to the user through a graphic display.

B. Data calibration

The magnetic field measurements collected by the manumeter are directly influenced by movement of the ring. Thus, by processing these measurements, it is possible to generate estimates of wrist flexion/extension, wrist radial and ulnar deviation, and finger flexion and extension angles (about the metacarpo-phalangeal joint of the finger wearing the ring). This process involves taking a differential measurement from the two magnetometers and then using a radial basis function network to map the differential signals to their corresponding joint angles.

1)Taking the differential signal

Because the strength of the ring’s magnetic field measured at the watch is comparable to that of the earth, it is necessary to cancel out the effects caused by the earth’s magnetic field. Since the earth’s magnetic field does not change much over short distances whereas the field of the magnetic ring does, this is achieved by taking a differential signal between the two magnetometers (roughly 3cm apart). Before taking the differential signal, the magnetometers are first calibrated and registered to one another to ensure that they respond uniformly. This calibration is performed using the earth’s magnetic field as a reference as described by [24]. Since the earth’s magnetic field should not change appreciably over the course of the calibration, sampling this vector while moving the manumeter through many different orientations should sweep out a spherical cloud of data-points in the magnetometers’ input space. However, small magnetic distortions caused by the battery and ferromagnetic components located on the data-logger board cause the magnetometers to respond more strongly in some directions than in others. These distortions cause the data to form point-clouds that can be closely modeled by ellipsoids [24]. By fitting an ellipsoid to the data collected by the front magnetometer, the necessary transformation needed to undistort the data back into a sphere can be found [24], [25]. Once the front magnetometer has been calibrated using this method, the affine transformation matrix needed to translate, scale, and rotate the data from the second magnetometer such that the difference between the two magnetometers is minimized is then calculated using linear least-squares optimization.

2) RBF network for joint angle estimation

Once the differential signals have been obtained, a radial basis function (RBF) network is used to map these signals to joint angle estimates (Fig. 3) [27]. To train the RBF network, a short dataset is first collected in which magnetometer measurements from the manumeter and their corresponding joint angles as measured using a custom-designed passive exoskeleton are collected simultaneously. Twenty five radial basis functions are then assigned to the network and trained using a combination of supervised and unsupervised learning processes.

Fig. 3.

RBF network used to map magnetometer values to joint angles. The network takes the x, y, and z components of the magnetometer differential signal as inputs. These inputs are sent through a bank of 25 radial basis functions and then weighted and summed to produce estimates of radial/ulnar deviation, wrist flexion/extension, and finger flexion/extension as outputs.

Centers for the RBFs are found by using a k-means algorithm to identify cluster centers in the manumeter’s input space [26], and the widths of the RBFs are set to the average distance from each RBF to its 10 nearest neighbors. Once the RBFs have been placed and shaped, their weights are then found using a variation of linear least-squares optimization known as ridge regression (or weight decay) [27], [28]. Using ridge regression, the squared error between the training signals collected from the exoskeleton and the output of the network is minimized as it would be using linear least squares, however, an additional term is added to the optimization to penalize large weights. This allows the flexibility of the model to be controlled thereby reducing over-fitting effects [27]. The contribution of the weight penalty to the optimization is controlled by a regularization factor which is selected to maximize generalization capabilities of the model by using a simple re-estimation formula [28].

3) Estimating hand use: angular distance traveled

Our initial approach to quantifying use of the hand is to estimate the angular distance the wearer moves in wrist flexion/extension, wrist radial and ulnar deviation, and finger flexion and extension angles throughout the course of a day. To estimate the distance traveled in each degree of freedom, the angular velocity of each angle estimate is first found using a discrete derivative. Distance traveled is then calculated by taking the integral of the absolute value of the angular velocity as shown in equation 1.

$$\mathit{dst}={\int }_{t_{\mathit{start}}}^{t_{\mathit{stop}}}\mid \omega \mid \mathit{dt}$$ (1)

This process is equivalent to summing up the degrees moved between each sample.

4) Proof of concept experiment

To demonstrate that the manumeter can be used to track movement of the wrist, hand, and fingers, we performed a series of tests on a single subject. The data collected by the manumeter were compared to data collected using the calibration exoskeleton. Prior to the first test, the magnetometers were calibrated and the RBF network was trained using the methods described above. Immediately after calibration, a 15 minute trial was performed in which joint angle estimates from the manumeter and direct joint angle measurements from the exoskeleton were collected simultaneously. To ensure that the data collected from both devices were properly synchronized, the devices were sampled by a host computer at regular intervals. During this 15 minute period, the subject was instructed to explore the full range of motion of his wrist and fingers. The manumeter and exoskeleton were then removed and replaced without recalibration, and a second dataset of one hour duration was collected.

To determine the accuracy of the manumeter, the manumeter estimates of wrist radial/ulnar deviation, wrist flexion/extension, and finger flexion/extension were combined with their corresponding direct exoskeleton measurements to form three N by 2 datasets (where N was the number of datapoints). Principal component analysis was then used to find the primary and secondary directions of variance in each set. Data oriented along the primary direction were taken to represent the amount of signal in the dataset, and data in the second direction were taken to represent the amount of noise in the dataset. The range of angle estimate errors enclosing 95% of the data was then found by taking twice the standard deviation of the data in the error direction. To identify changes in hand use over the course of the measurement period, the distance traveled by the hand was evaluated within one minute bins.

III. Results

To demonstrate the potential of the manumeter as a device for quantifying hand use, we collected estimates of wrist and hand movement from a subject wearing the manumeter and compared them to estimates collected using a custom-designed passive exoskeleton. We performed a 15 minute trial immediately after calibrating the manumeter and an hour long trial after removing and replacing the manumeter without recalibration. For the 15 minute trial, 95% of the angle estimates for wrist radial/ulnar deviation, wrist flexion/extension, and finger flexion extension were found to be within +/− 1.4, 6.4, and 4.7 degrees respectively, of the “gold standard” values from the exoskeleton. For the one hour trial, we did not recalibrate the device and instead used the calibration performed before the 15 minute trial. For this longer trial, 95% of the estimates were found to be within +/− 2.4, 5.8, and 4.7 degrees respectively (Fig. 4). Estimates of the distance traveled in radial/ulnar deviation, wrist flexion/extension, and finger flexion/extension during the hour long test as measured by the manumeter were significantly correlated with estimates taken using the exoskeleton (P < .001 all, R² = 0.90, 0.89, and 0.92 respectively Fig. 5). The total distance traveled in radial/ulnar deviation, wrist flexion/extension, and finger flexion/extension as measured by the manumeter were within 10.4%, 4.5%, and 14.3 % of their actual values as measured using the passive exoskeleton.

Fig. 4.

(Left column) Angle estimates from the last three minutes of a 1 hour test of the manumeter. Estimates taken using the manumeter are shown in solid dashed red, and estimates taken with the manumeter are shown in solid blue. (Right column) Correlations between the two measurement devices. The minor axes of the ellipses fit to the data enclose 95% of the data in the error direction as defined using principal component analysis (r = radial, u = ulnar deviation; f = flexion, e = extension).

Fig. 5.

(Left column) Distance traveled estimates taken using the data from the passive exoskeleton (wide blue) and the manumeter (narrow green). Plots in the right column show the correlation between the estimates taken from the manumeter and the estimates taken from the passive exoskeleton. (Right column) Correlation coefficients for the distance traveled estimates of wrist radial/ulnar deviation, wrist flexion/extension, and finger flexion extension were 0.97, 0.90, and 0.8 respectively

IV. Discussion

This paper provides a proof of the concept that movement of the magnetic field produced by a magnetic ring worn on the finger can be detected at the wrist and processed to estimate wrist flexion/extension, radial/ulnar deviation, and finger flexion extension about the metacarpo-phalangeal joint. We show here in a pilot study that the manumeter can be used to reliably estimate joint angles and total angular distance traveled in a one hour trial. By using a permanent magnet to produce signals at the finger and a pair of magnetometers to receive those signals at the wrist we have created an untethered sensing mechanism that can be incorporated into common, socially accepted accessories (a ring and a watch-like band).

It may be possible to further improve the resolution of the device. Errors in the angle measurement are sometimes exhibited as offsets (Fig. 4) that appear to be due to metal components on the PC board in the watch-like unit. These metal components distort the ring’s magnetic field and can become magnetized over time, causing the raw magnetometer signals to slowly drift as guided by the previous history of movement of the magnet. Such errors can be reduced in future designs by increasing the distance between metal components and the sensors. However, we note that the distance traveled measurement is already relatively insensitive to this drift, as the changes in the resulting offsets were small relative to the total distance traveled within an epoch.

The results of the proof of concept test described here need to be verified with more subjects, both unimpaired and impaired, who wear the device for a longer duration in the real world. A key issue for future research is the effect of environmental ferromagnetic material, such as elevators, car doors, or metal utensils on the device readings. It should be possible to identify close interactions with metal because such interactions often cause the magnetometer readings to vary in ways not possible due to anatomical movement of the magnetic ring alone. Important safety considerations include avoiding close proximity to sharp ferrous objects, MRI machines, and other strong electromagnets, and analyzing possible effects on pacemakers.

We envision the manumeter aiding clinical practice and research in several ways. Healthcare providers could use the manumeter to gather objective information about the user’s daily movement habits to administer appropriate, personalized therapy. For research, the manumeter may improve estimates of how much subjects use their impaired limbs in their daily lives. Unlike estimates of spontaneous use obtained using accelerometry, the manumeter can convey detailed, quantitative information regarding how the wrist and hand are actually being used.

The manumeter may also be useful as a therapeutic tool, as it could be used to provide a daily report of hand and arm use that informs the user whether he or she is meeting daily movement goals. Simple forms of feedback about movement performance can improve recovery of motor function in people with stroke [29]. In conjunction with daily movement monitoring, the manumeter could also be used as a real-time input device for therapeutic video games to provide supplementary movement practice. The device might also be useful for measuring wrist and hand posture in the real world, in order to gain insight into use-related injuries such as carpal tunnel syndrome.