Wearable motion sensors to continuously measure real-world physical activities

Abstract

Purpose of review

Rehabilitation for sensorimotor impairments aims to improve daily activities, walking, exercise, and motor skills. Monitoring of practice and measuring outcomes, however, is usually restricted to laboratory-based procedures and self-reports. Mobile health devices may reverse these confounders of daily care and research trials.

Recent findings

Wearable, wireless motion sensor data, analyzed by activity pattern-recognition algorithms, can describe the type, quantity, and quality of mobility-related activities in the community. Data transmission from sensors to the cell phone and Internet enable continuous monitoring. Remote access to laboratory-quality data about walking speed, duration and distance, gait asymmetry and smoothness of movements, as well as cycling, exercise, and skills practice, opens new opportunities to engage patients in progressive, personalized therapies with feedback about performance. Clinical trial designs will be able to include remote verification of the integrity of complex physical interventions and compliance with practice, as well as capture repeated, ecologically sound, ratio-scale outcome measures.

Summary

Given the progressively falling cost of miniaturized wearable gyroscopes, accelerometers, and other physiologic sensors, as well as inexpensive data transmission, sensing systems may become as ubiquitous as cell phones for health care. Neurorehabilitation can develop these mobile health platforms for daily care and clinical trials to improve exercise and fitness, skills learning, and physical functioning.

Introduction

Mobile health or mHealth is a growing endeavor to improve healthcare services via mobile communication devices.¹ The cell phone enables continuous access to the Internet over broadband and WiFi for data transmission of physiologic variables, physical activity, blood tests, images, social interactions, mental states, and environmental conditions.² By simultaneously assessing behavioral, physiological, and psychological states in the real world and in real-time, mHealth also aims to quantify states of health and well-being. Feedback, cues, and updated instructions via graphics and text messages can be provided in real time based on the flow of information from and back to a patient. The result will be high throughput, multi-streamed, longitudinal data sets to facilitate disease prevention, diagnostics, compliance, personalized management, and behavioral change.³ A global aim is to use this technology to reduce healthcare disparities, especially for patients with chronic diseases, and lower the long-term cost of more personalized care. This long-term management capability is especially important in neurologic rehabilitation after disabling spinal cord and traumatic brain injury, as well as in stroke, multiple sclerosis, and any progressive or neurodegenerative disease. Thus, the rehabilitation team may find remarkable opportunities in mHealth, just as it has for other assistive technologies.

Mobile health smartphone apps take advantage of external sensors and the camera, microphone, GPS, and accelerometer built into these communication devices. The phone already serves as a transmission relay for Bluetooth equipped weight scales, blood pressure and heart rate devices, equipment for exercise, and mental and social health state assessments. Bio-monitoring of blood chemistries, embedded lab-on-a-chip sensors, and tele-monitoring for remote personal health advice by professionals are moving forward as well. Evidence for efficacy is growing, if slowly.⁴ For example, the first mHealth Cochrane analysis of randomized clinical trials (RCTs) for self-management of type 2 diabetes found larger effects on glucose and HgbA1C control for cell phone-based interventions compared to conventional information and computer use.⁵ Studies of efficacy, however, are sparse. Across all health conditions at the end of 2012, 176 RCTs of mHealth technologies were listed at clinical trials.gov,⁶ but few have been published or relate to neurologic disability.

This review describes efforts to bring wearable, wireless sensor networks to bear on community-based assessments and treatments to improve walking, exercise, fitness, and other mobility-related activities after neurologic injuries and diseases. It addresses the challenge of a white paper⁷ from the National Institute of Child Health and Human Development, which concluded, “Advanced technology/sensors must be developed to establish better tracking of compliance and clinical outcomes, at several International Classification of Functioning, Disability, and Health levels. New, low-cost, portable sensors may ultimately replace prevailing clinical instruments used for outcome assessments.” Inexpensive smartphones and tablets are lowering the complexity of this challenge since they can communicate with multiple sensors placed on the body; initiate, store or transmit data for processing; provide a variety of user interfaces; download instructions and reminders; and remotely update applications.

Sensor Platforms

A wide range of wearable sensors (Table 1) are available commercially that provide the raw data to describe arm, trunk, and lower extremity actions outside of a motion analysis gait laboratory.⁸ The choice of sensors, number, and placement will depend on the activity and movement variables to be ascertained. Practical sensor systems must meet many complex design requirements, from cosmetic, privacy and technology acceptability by users to signal processing, data transmission, annotation, and scalability for easy use (Table 2). Especially important for motion sensing is the accuracy and speed of feature detection and classifier algorithms that turn a sequence of inertial signals into a recognizable movement pattern to measure clinically important details of gait and other purposeful activities.

Table 1. Types of wearable sensors to assess physical activity.

Triaxial accelerometer: accelerations/decelerations, velocity and displacement of a body segment in x, y, z axes.

Gyroscope: angular velocity and rotation.

Global positioning satellite (GPS) signal: location primarily outdoors; may calculate speed and distance of continuous walking with smartphone app.

Magnetometer: directional vectors of spatial orientation.

Electromyography: dry electrodes for surface EMG of timing and amount of muscle group activation.

Goniometer: joint angular range of motion.

Resistive flex and pressure sensing: fiberoptic or deformable textile across a joint detects angular change; piezoelectrode for distribution of weight on sole to define stance in the gait cycle.

Environmental context: ambient sound, light, motion-activated photo or video.

Table 2. Technical features for practical remote motion-sensing systems.

Sensors:

Type, number and position depend on specific body metrics sought

Design – e.g, piezoelectric or capacitive microelectro-mechanical-system accelerometer

Cosmetic acceptability; ease and reproducibility of placement.

Raw signal structure and sensitivity to events

Firmware instructions for device components

Partial data processing on sensor chip

Platforms:

Interoperability by using common software, communication, data processing and confidentiality protocols

Open source, publically available standards

End-to-end system reliability

Data transmission:

Choice of wireless standards – Bluetooth, Zigbee, Wi-Fi, voice channels, Short Message Service, Universal Mobile Telecommunications Systems

Cost

Frequency of data sampling

Bandwidth

Power consumption; energy source

Reliability

Data time stamping

Error check

Storage capacity

Secure data at each stage of collection, transfer, and storage

Signal processing:

Temporally fuse data synchronously from multiple sensors and body sites

Analytic algorithms

Features assessed include mean of signal, peak frequency, correlation of axis, signal energy, standard deviation

Classifier models include naïve Bayes, support vector machine, decision tree, hidden Markov, neural networks, spectrum analysis, random forest

Integrate multiple layers of the classifier, e.g., activity, context, sensor location

Artifact recognition; examine outliers

Environmental context of activity

Speed of processing

Machine-learning analysis

Resolution of data:

Software to interpret data from sensors and other sources of information to provide new insights into health states

Normalized for matched population and sensitive to individual's daily functioning over time

Discern trajectory of change and clinically meaningful gains and declines

Visualize data using customizable tools and reports

Annotation:

Describe changes in health, mood, behavior, social circumstances, environment

Ontological encoding of data across studies, e.g., Unified Medical Language System for standard description of medical condition, treatments, responses and contexts

Methods to scale up applications:

Simplify instructions, minimize time and effort by user; keep cognitive load low

Minimize steps and increase automaticity in data flow during acquisition, processing, analysis and search

Conceptualize summary data for practical uses, such as feedback, monitoring and outcome tools

Data accessibility in common databases:

NIH or Research Electronic Data Capture (REDCap) databases

Annotated raw data repository for data mining

Data privacy and security:

Encryption

HIPAA requirements

Commercial devices

Recently, fitness, exercise and wellness gadgets have come to the social networking market. Can they be used for patient care? In general, these cosmetically striking devices detect successive movements by a single biaxial or triaxial accelerometer placed in a pocket or on a wristband (e.g., FitBit, BodyMedia, FuelBand). Results are summarized by downloading data to a computer or smartphone usually via Bluetooth. Episodic and cyclical body movements are then calculated as activity or step counts or converted into calorie counts. Each swing of the arm or forward propulsion of the trunk is interpreted as a stride during repetitive exercise. Actions with low gravitational force or unusual combinations of acceleration-deceleration of short duration may be misinterpreted, however. Adventitious movements may be interpreted as the motion of interest. Reliability and validity are uncertain in healthy persons in real-world settings and yet to be studied in disabled persons. At best, a wrist-worn accelerometer may distinguish sedentary, household, walking and running as distinct activities and correctly classify intensity of activity 50% of the time.⁹ In their present configuration, these are not suitable for research on patients with neurologic impairments.

Single accelerometer-based step counters have been available for 2 decades for outpatient use (e.g., Actigraph, Pensacola FL; StepWatch Activity Monitor, Oklahoma City, OK).^10,11 Their count of steps over time generally correlate with the degree of walking impairment for patients with stroke (e.g., slower walkers take fewer steps)¹² and other neurological diseases. Like even less sophisticated pedometers, they may not detect all steps when the cadence falls below 50/minute, walking speed slows below 0.6 m/s¹³ or the gait pattern includes irregular movements. None measure walking speed or have yet been enabled to download to a smartphone. Triaxial accelerometer systems placed posteriorly at the midline of the waist use proprietary algorithms to detect the gait cycle and walking speed (e.g., Actibelt, Munchen, Germany), but so far, tend to be less accurate in patients with greater impairment who walk slowly.^14-16 Indeed, multi-sensor systems are significantly more accurate than any of these single accelerometers to measure activity and estimate energy expenditure.¹⁶

Research devices

An important goal for rehabilitation is to be able to remotely classify human activities and quantitatively measure the quality of their component movements outside of a motion analysis laboratory. Wireless gait laboratory systems (e.g., APDM, Portland, OR) that integrate from 2-7 accelerometers and gyroscopes worn on the wrists, ankles and chest or waist, plus additional types of sensing, are said to be accurate for revealing the gait cycle and walking speed. Combinations of accelerometers are also sufficient to detect postural imbalance,¹⁷ and may help detect or predict falls. Wheelchair activity and energy consumption measurement also requires multiple sensors, on each arm and the chair.¹⁸ These systems, due to cost and complexities in management, have primarily been used in controlled settings, but not for continuous community usage enabled by automatic downloading to a smartphone.

Comfortable, user-friendly sensor network designs compatible with the notion of mHealth are becoming available.¹⁹ In one study, low-cost, miniaturized triaxial accelerometers with electronic circuits were placed over the tibia just above both ankles in healthy and hemiplegic participants. A template walk at several speeds for 10 m was used to help train the activity-pattern-recognition algorithm for each subject.²⁰ The synchronous bilateral raw inertial signals were examined for features related to the timing of components of each stride, including heel-off, toe-off, peak swing, end of swing, and foot flat. A machine learning, Bayesian activity-recognition classifier was developed that grouped activities and set the features that distinguished them. The algorithm then recognized subsequent bouts of walking across a day's activity and calculated walking speeds in the stroke patients as low as 0.1 m/s, along with distance and duration of each bout, and limb asymmetries in stance and swing times. This protocol led to high correlation with ground truth measures during walking in the community.^20,21 This sensor and analysis system was then used to provide feedback over the Internet about daily walking bouts in terms of speed, duration and distance in a RCT during inpatient stroke rehabilitation at 15 sites in 12 countries.²² Over 2100 hours of activities were identified and quantified in 140 subjects, revealing the progression of walking-related measures and the actual amount of physical therapy provided for mobility. A Bluetooth connection from the sensors can download the data to a smartphone as well, then to a remote server for algorithm processing. Another research group placed bilateral accelerometers at mid-leg along with a gyroscope to try to eliminate the template walk, but their algorithm was only accurate when walking speed exceeded 0.6m/s.²³ Other sensor placements and approaches to feature extraction from the accelerometer signal have been reported for subacute stroke,²⁴ Parkinson's,²⁵ and multiple sclerosis.¹⁷

Thus, much progress is being made for personalized motion technologies. A smartphone with a continuously running software application that compresses and transmits data to a central server can be an effective hub to manage multiple streams of sensor and other physiological data.²⁶ Practical sensing for the study of patients, however, requires technical and logistical development and planning.² In addition to features listed in Table 2, cultural acceptance of technologies must evolve to optimize utilization. For inexpensive, wide utilization, interoperability of software and communication systems, publicly open standards, and qualitative and quantitative evidence about what works for what population under specified conditions seems essential.^4,27 For neurology and rehabilitation, efficacy and effectiveness trials are necessary before a final iteration of hardware, software and infrastructure should be scaled for wide usage.

Motion Sensing for Daily Care

Disabled persons, such as those after stroke, take far fewer steps daily, with fewer and shorter bouts of walking compared to healthy peers.²⁸ Critical research to understand how to reduce risk factors for vascular disease, for example, and to reduce disability and increase daily participation will benefit from the ability to quantify the type, quantity, and quality of daily activities.⁸ Sensor networks that monitor upper²⁹ and lower extremity²⁰ activities should facilitate accurate ongoing assessment during community functioning and enable frequent recommendations about how to progress exercise and skills practice from remotely located professionals. Sensors, then, may alter behavior by offering feedback and personal activity auditing that encourages self-efficacy in the form of graphics and instruction from anywhere the Internet reaches. When particular exercises and skills practice are prescribed during long-term rehabilitation efforts, both patients and caregivers may benefit from remote supervision that addresses their concerns about safety and how best to work to advance the reacquisition of skills.

Although this level of monitoring could be viewed as an invasion of privacy, disabled persons are likely to applaud the accessibility of rehabilitation supervision in the context of their home and community at low cost. Tele-neurology³⁰ and tele-rehabilitation³¹ could interface with wearable sensor technology to complement home-based care and compliance with medical recommendations.

Sensors for Clinical Trials

Having ground truth about activity levels, in terms of frequency, duration, intensity, and energy consumption, will turn assumptions about the quantity of exercise and practice during trials into certainties. For example, all of the large recent RCTs of treadmill and robotic training to improve walking after stroke,^12,32-34 spinal cord injury,^35,36 Parkinson's,^37,38 and multiple sclerosis³⁹ have assigned subjects in the control and experimental groups to a specified number of hours of weekly treatment. None of the studies, however, can report with confidence how much walking and exercise occurred during planned practice sessions or whether participants practiced locomotor skills and exercised outside of formal training times.⁴⁰ Exercise trials that take place in the community are even less likely to be able to capture the quantity of practice.^41,42 Yet a bias toward high or low levels of practice beyond what the investigative team sees may have a confounding impact on the effects of the experimental therapy. For example, participants who practice more may gain better skills; incorrect practice could reduce the effect of the formal therapy. The quantity and quality of an experimental physical intervention may also vary across the multiple sites of an RCT or change when a new therapist replaces the one who was trained at onset of the trial. Good trial design recommends that extensive training in provision of a complex physical intervention take place before an RCT starts and that videotaping of the intervention or in-person, intermittent monitoring be part of the protocol at subsequent intervals. The conventional approach to these monitoring needs may be less reliable and cost more than intermittent remote sensor monitoring of actual practice (how much, how well) during formal training sessions and in between therapies.

Continuous monitoring of what subjects actually perform enables other benefits to trial integrity and design. Serial sensor measures can provide dose-response assessments or be used for imputation by statisticians when a participant drops out. Real-world sensing also offers ecologically sound, interval and ratio scale assessments to augment questionnaires and ordinal scales about disability, participation in fulfilling personal goals and roles, and physical functioning (Table 3). Quality of life tools for this have become a requirement as primary or secondary outcomes in neurologic trials. Most diseases have their own tool, often derived from questions developed for the Medical Outcomes Study's SF36 and now represented in the NIH's NeuroQOL toolbox.⁴³ These Likert-scaled measures of change in daily physical activity and ratings of difficulty (climbing stairs, walking 1 block, etc), however, have usually not been confirmed by real-time studies of these activities. For example, the reported level of independence by persons with SCI differed from what clinicians found on testing.⁴⁴ Wearable sensors can provide that ground truth.

Table 3. Comparison of conventional scales and wireless, wearable sensor-derived tests of mobility-related functioning.

DATA	USUAL METHOD	mHEALTH SENSORS
Type of physical activity	Self-report diary or checklist; observe in lab; video; short distance timed walk or distance walked in 2-6 min.	Activity pattern-recognition algorithms; walk, cycle, leg exercises identifiable by sensor data processing
Quantity Frequency/duration	Observation; inertial movement/step counts if accelerations high enough	Directly measure wave forms of individual components and whole actions
Quality	Laboratory motion analysis or pressure mat system	Compare each leg during step cycle in context of environs
Location of activity	Self report; lab	Anywhere; global positioning & ambient context sensing for site identification
Reliability	Inter-rater; test-retest	Ground truth measurement v. sensor-based algorithm
Validity	Content/construct for each scale	Face validity; responsiveness
Statistical testing	Ordinal scales of physical functioning	Interval / ratio scale data
Data entry	Computer	Smartphone, tablet
Human factors	Train examiners in test administration	Train participants in a culture of technology
Regulation	Local Institutional Review Board and HIPAA	Local IRB & HIPAA; possibly Food and Drug Administration

Just as self-reporting scales stand as a partial surrogate for actual activity and participation, so do other commonly used walking-related outcome tools, such as the timed short-distance walk (6-15m) and the distance walked in 2-6 min in a laboratory setting. In general, improved effects on surrogates do not necessarily transfer into health benefits; indeed, the surrogate may fail as a guide to the most clinically meaningful and effective therapies.⁴⁵ In neurorehabilitation trials, a pre- to post-test gain of >20% in 10-m speed or 6-min distance often reaches statistical significance and favors one intervention over another. The clinical meaningfulness of such change, however, is uncertain. The gain may generally correlate with self-reported functional measurement tools,⁴⁶ but outliers are common, because reliability of self-reports are uncertain. The ability to serially capture walking-related variables in the home and community, to examine changes in speed and leg symmetry on varied surfaces, and capture changes in exercise capacity, for example in relation to pain, fatigue or adverse effects of medications, should provide greater insight into the effectiveness of new therapies in all patients for whom an evidence-based trial suggests efficacy.^47,48

The frequency at which patients might be monitored by wearable activity-sensing networks depends on the object of the study. Levels of walking activity using pedometers require about 7 days of data collection to obtain a stable and representative average for healthy persons⁴⁹ to as little as 2 days for those with incomplete SCI.⁵⁰ For a clinical trial of a walking intervention of 3 months duration, a minimal data set might include 2 weeks of daily monitoring prior to starting the comparison treatments, then for one week monthly or at the time of scheduled outcome measures. For a drug trial, activity might be measured continuously for at least a month – two weeks prior and at least 2 weeks after initiation to detect fluctuations in response to medications (e.g., dyskinesias or freezing of gait in Parkinson's disease, leg spasms in SCI). Skills practice at home might be assessed for 1-2 sessions a week to monitor quality of movements. Schedules for feedback about performance to motivate compliance will have to be empirically derived.

Conclusion

Wireless remote sensing to monitor the type, quantity, and quality of physical activities, daily participation, and skill reacquisition offers great potential for neurologic and neurorehabilitation patient care and clinical trials. Progressive reductions in the cost, size and energy requirements of gyroscopes, accelerometers, other physiologic sensors and data transmission over the Internet, along with empirical work on activity-recognition algorithms, suggest that wearable systems may become ubiquitous tools. Efficacy and effectiveness trials are necessary, however, before clinicians can utilize sensor data for ecologically sound monitoring and outcome measures.

Key points.

Wireless inertial and motion sensor devices, worn on the arms, trunk or legs, can send raw data over the Internet to reveal daily activities.

Sensor-derived, activity pattern-recognition algorithms are being developed to identify the type, quantity and aspects of quality of purposeful movements.

Data about walking speed, distance, duration and gait asymmetry, as well as exercise, can be used to provide remote feedback about practice and skills learning in the context of the home and community, as well as for ratio-scale outcome measures.

Future improvements in access to rehabilitation care at low cost may be made feasible by the combination of wireless broadband networks, ubiquitous penetration of cell phones, and wearable technologies for personal and environmental sensing.