Reconstruction of body motion during self-reported losses of balance in community-dwelling older adults

Abstract

Older adults experience slips, trips, stumbles, and other losses of balance (LOBs). LOBs are more common than falls and are closely linked to falls and fall-injuries. Data about real-world LOBs is limited, particularly information quantifying the prevalence, frequency, and intrinsic and extrinsic circumstances in which they occur. This paper describes a new method to identify and analyze LOBs through long-term recording of community-dwelling older adults. The approach uses wearable inertial measurement units (IMUs) on the feet, trunk and one wrist, together with a voice recorder for immediate, time-stamped self-reporting of the type, context and description of LOBs. Following identification of an LOB in the voice recording, concurrent IMU data is used to estimate foot paths and body motions, and to create body animations to analyze the event. In this pilot study, three older adults performed a long-term monitoring study, with four weeks recording LOBs by voice and two concurrent weeks wearing IMUs. This report presents a series of LOB cases to illustrate the proposed method, and how it can contribute to interpretation of the causes and contexts of the LOBs. The context and timing information from the voice records was critical to the process of finding and analyzing LOB events within the voluminous sensor data record, and included much greater detail, specificity, and nuance than past diary or smartphone reporting.

I. INTRODUCTION

Losses of balance, LOBs, defined as events where balance is lost momentarily but recovered, are more common than falls and are closely linked to falls and fall injuries. LOBs have long been identified as common, occurring at least once over a three week period in 35% of community older adults [1]. Trips and slips are major risk factors for falls in older adults and can lead to injury including fracture [2][3][4]. Multiple stumbles and missteps also increase the risk for falls [5][6]. In laboratory settings, it has been demonstrated that the inability to recover from a LOB is an independent predictor of a future fall [7].

Despite this frequent incidence and connection to falls, little is known about the biomechanics of real-world (i.e. outside the laboratory) LOBs or the context in which they occur. Real-world LOBs are particularly challenging to identify and record. Although indicative, falls diaries are susceptible to underreporting and poor recall [8] and, aside from laboratory simulations [9], there is little quantification of what actually occurs during real-world slips, trips, or other LOBs. In one successful case, the subset of LOBs that result in falls was captured in an exhaustive video record of the public areas of a long-term care facility [10]. Yet, this video-recording approach applies only to clearly identifiable LOBs and falls in a video-monitored location, and still provides little information beyond qualitative classification of activity and mechanism. Little is known about real-world LOBs that do not result in falls, including their causes, contexts, and the strategies used to recover. Thus there is a need to study real world LOBs, both their biomechanics and the context in which they occur [11][12].

Wearable sensors can be used to quantify sit to stand transitions [13] and key measures of stepping and walking [14][15][16][17]. However, these methods work best during consistent repetitive motions with a continually changing base of support, such as during a long bout of walking (e.g. 60 seconds [18]), rather than episodic or intermittent events such as turning or reaching. Other devices with multi-axial or additional sensors can detect specific activities such as standing or sitting [19], and simulated falls [20][21], where there is a very distinct and recognizable pattern. Algorithms based on laboratory-induced LOBs may be applied to identify LOBs in the real world [22]. Yet, a rigorously defined laboratory-based LOB may not represent real life events. Furthermore, movement patterns in LOBs that do not result in a fall can be both subtle and complex.

Given the potential diversity of real world LOBs, one solution is to gather kinematic data with an array of sensors, such as with inertial measurement units (IMUs, comprising accelerometers, gyroscopes and optionally magnetometers) worn on key body segments. Multi-sensor body tracking systems (e.g. XSens BV, Netherlands) can reconstruct body position based on a kinematic chain of limb segment orientations. This skeleton-based approach can accurately place segments in relation to each other, but it requires a dedicated sensor for each articulated segment – typically 15 or more for the whole-body – rendering the systems complicated to wear and therefore unsuitable for unsupervised daily tracking.

The present approach is based on an estimated reconstruction of the body’s movement from a reduced set of sensors, which is not dependent on a detailed skeleton model. This approach begins with estimating the trajectories of the feet (3D position in addition to orientation) by mathematically integrating the IMU signals. Then, assumptions about movement are used to co-locate the two feet, the trunk and one arm in order to produce a data-driven animation of whole-body motion. The resulting four-IMU approach provides motion reconstruction of the segments most critical for balance loss and recovery, i. e. the feet and trunk.

To find the rare LOBs within volumes of sensor data, we further propose an improved approach for gathering information on the occurrence, context and mechanisms of LOBs. This approach uses wearable voice recorder on which participants identify, explain and time-stamp LOBs as they occur. This context information, combined with the context-specific reconstructed motion from the LOBs, can identify LOBs and recovery strategies in specific real world situations that have previously been unclear to both the observer and the individual.

The goal of this work is to build the capacity to study the contexts, perceived causes, and movements associated with both recovered and unrecovered LOBs (i.e. falls), particularly in older adults who are at higher risk for falls and falls-related injury. We demonstrate a method for long-term, field-based studies that builds a profile of each LOB by recording its context and reconstructing its motion. We propose that these LOB profile data may eventually lead to: 1) expanding fall and injury risk assessment by identifying behaviors and movement patterns that risk inducing LOBs in specific individuals; and 2) informing fall prevention programs to provide task-specific balance retraining, increase self-awareness of balance limitations, and provide guidance for environmental modifications. This paper presents the methods for profiling LOBs and a case series demonstrating the value of combined contextual and movement information.

II. Methods

We recorded body motion and LOB context information during daily activities in three healthy independent community dwelling older adults (2 male and 1 female, aged 66, 73 and 71 yrs.). The University of Michigan IRB approved the study and all participants provided informed consent. Participants were recruited with the assistance of the University of Michigan Older Americans Independence Center Human Subjects and Assessment Core.

The proposed new method incorporated five key components. First, wearable IMUs were mounted on each shoe, the trunk and one wrist to provide a summary of movement preceding, during, and following the LOBs. Second, a wrist-worn voice recorder was used to provide an immediate and accurate record of the LOB context and timing. Third, the quantitative information captured by the IMUs and the voice record time-stamp were used to identify a time window surrounding the LOB. Fourth, reconstruction of body kinematics was used to create body motion animations and pinpoint the exact location of the LOB. Finally, the voice recording, the body animations, and the kinematic measurements were used to analyze the LOB. All participants wore the voice recorder for 4 consecutive weeks and the IMUs during the 2nd and 4th weeks.

A. Wearable IMUs

Participants wore four commercial IMU data loggers (Opal, APDM Inc., Portland, OR, USA; synchronized 128 Hz sampling, +/−6G acceleration, +/−2000 deg/sec angular rate) to capture raw acceleration, angular rate, and magnetic field orientation. The IMUs were worn on the feet using pouches attached to the tops of each shoe, on a waist belt (at the center of the back), and on a wrist strap. Participants were given instructions on how to put on the sensors each day and how to move sensors among different pairs of shoes, including slippers with custom pouches sewn on top. Each IMU was marked with a specific location (left foot, right foot, waist, wrist) and each pouch had a diagram showing correct sensor insertion. However, subjects did not need to follow an IMU pose calibration or a strictly reproducible placement procedure, because our movement reconstruction algorithms can use the IMU information regardless of their precise location - the main requirement was that each IMU be on the correct segment. Participants were to wear the IMUs each day from the time they rose in the morning until bedtime at night, excluding bathing, during all of their in-home and community activities.

B. Voice recorders

Participants reported the context and approximate timing of LOBs on a watch-sized voice recorder (Sansa Clip Zip 8GB MP3 player, with modified RockBox open source software and custom wrist strap). The recorder was programmed to perform only one function: to record time-stamped voice data for two minutes upon the press of a single button. The recorder, synchronized to the IMUs at study onset, retained synchrony to within about 1 minute per month and maintained battery power for the study duration. Participants were to record a description of any LOB, including its context, as soon as possible following each event. As per our previous work [23][24], LOBs were defined as events where balance control is lost at least momentarily, such as a slip, trip, stumble, or misstep. A righting response occurs subsequent to many LOBs, and in some cases a fall ensues.

C. LOB time-window identification

A typical LOB event may last only a few seconds, yet must be identified and localized from 12 or more recorded hours. The time-stamps of the voice recordings were used to locate the corresponding sensor data, typically within a 0-5 minute window preceding the voice recording, or at the time reported in the participant’s description if different.

D. Foot-path estimation and body motion animation

For each LOB, sensor data were processed to estimate the motion of each foot and the orientations of the trunk and arm. Foot trajectories were calculated using a magnetometer-free inertial reconstruction algorithm [16][17][25] validated to provide accurate foot motions as well as outcome measures such as stride variability [14][26] and locomotor task performance [27][28][29]. These two foot trajectories were combined with trunk and arm orientation estimates to generate an estimated body pose. The pose estimate is based on assumptions about the basic structure of upright bipedal locomotion. The key in applying this method to movement analysis is to measure aspects of movement that are not sensitive to error in these assumptions.

First, the two feet are assumed to walk in roughly the same direction, and to be positioned in a lateral direction relative to each other at the time of each mid-stance, at a particular average distance. This constraint was loosely enforced using a two-step correction on (1) heading direction and (2) distance between the feet (nominal value 0.3 m). The direction of walking from each foot and the mean heading across the two feet were computed, and each foot trajectory was rotated by a low-pass filtered version of its heading error (2^nd order zero-lag Butterworth filter; cutoff 0.1 Hz). The relative position vector between the feet at each footfall was then computed, and a constraint error was estimated by subtracting this vector from the assumed purely lateral foot separation. This vector error was low-pass filtered (same 0.1 Hz filter as above) and the result subtracted from the estimated motion of the feet (half from each foot). The assumed distance for each individual was adjusted to prevent collisions between the feet in the combined reconstruction. This alignment process resulted in slow adjustments establishing a regular walking pattern, but had little effect on the quick LOB irregularities and the resulting reconstruction.

Second, IMU estimated heading from the trunk and arm sensors (magnetometer-based proprietary algorithm from the manufacturer) was used to obtain a mutually consistent direction measured with respect to the global reference frame. A heading offset was then added to these segments to match the direction of walking according to the feet. This compliance was again reinforced by adding low-passed corrections to the heading orientations (same 0.1 Hz filter as above).

Third, the center of the base of the torso was assumed to be located at a certain distance (0.9 m) directly above the mean position of the two feet, and the torso was assumed to be upright on average during steady walking. Finally, the wrist sensor was assumed to be attached to a straight arm connected to a straight torso at a roughly anatomical shoulder location (shoulder 0.04 m forward, 0.20 m lateral and 0.57 m above the base of the trunk; 0.46 m directly proximal from the wrist).

In combination, these assumed adjustments resulted in large corrections to unify the otherwise disconnected trajectories of the independent sensors (e.g. left foot walking north while the right walks west). The reconstruction artifacts observed due to assumption errors were deemed acceptable because the goal was to illustrate the mechanisms of LOBs and balance recovery for qualitative interpretation by a human, rather than to estimate precise multi-segment kinematics. Individual segment kinematics (foot trajectories, trunk and arm orientations) remain accurate in isolation and could be further analyzed separately from the whole-body reconstruction.

E. LOB analysis

Around the time of the voice recording indicator of a LOB, the raw data and the reconstructed motion animation were used to investigate the kinematics of balance loss and recovery. The plots of each foot translational speed and location were particularly helpful in pinpointing the timing of the LOB. The foot speed has a stereotypical pattern during normal gait, reaching zero during each footfall and a peak during each swing phase. Unusual foot motions were quite apparent as deviations from the norm, such as: a sudden change in one foot’s speed interrupting the leg swing motion (tripping); simultaneous non-zero velocity of both feet at once (slipping); a crossover step or stutter-step; a footfall at a strange or awkward location; or an extended period of non-zero velocity. In other cases the feet did not cause the LOB, and the animation was used together with the signals from the trunk and arm to identify the LOB. A flowchart summarizing this process is shown in Figure 1.

Figure 1.

Flowchart showing the key components of the proposed method. IMU data (both feet, waist, one wrist) are collected continuously during daily life. Voice recorders (other arm) are used to provide LOB context. The time-stamp from the voice recordings defining a time-window when the LOB occurred. The foot trajectories are combined with trunk and wrist pose information to create full body animations, which help identifying the exact location of the LOB. Finally, the LOB is analyzed for cause and kinematic response.

III. Results

The three participants recorded 46 distinct LOBs during the four weeks (107 person-days total) of voice reporting, including 1 fall (0.43 LOBs per reporting day). Within this data set, 26 LOBs (0 falls) occurred during the two weeks (41 person-days total) when the IMUs were also worn. IMU data recording length averaged 12.9 ±1.5 hours per day (mean ± s.d.).

Four representative case studies of LOBs in the form of still frames from animation and accompanying data signals illustrate the event reconstruction process and outcome. Animations are included in the Supplementary Material. In the first case (Figure 2), participant #1 reported, “Walking in the parking lot—lost my balance—regained my balance by touching the car.” The corresponding data show a sudden change of the trunk orientation followed by a pattern of interrupted footfalls used to recover. This subtle LOB had no known cause, yet was alarming enough that the participant noticed and reported it.

Figure 2.

A loss of balance experienced by participant #1, with voice annotation, “Walking in the parking lot—lost my balance—regained my balance by touching the car.” (left) Data from IMUs. (right) Reconstructed body motions. The signals show a change in trunk pitch angle followed by a pattern of short steps used to recover. The cause of this LOB is not known, but a side-step and pause illustrates the recovery technique. See also the animation in supplementary material.

In another event (Figure 3), the same participant reported, “Lost balance when I was in by the computer—regained balance without fully falling.” In this instance, the participant did not indicate the source of the LOB; however, the data signals revealed some form of disturbance to the right foot. The associated animation shows that the right toe was caught on some stationary object, and the recovery was accomplished by a secondary high step over the obstacle with the right foot, followed by another step with the left foot. We also detected compensatory arm and trunk motion.

Figure 3.

Another loss of balance experienced by participant #1, with voice annotation: “Lost balance when I was in by the computer—regained balance without fully falling.” (left) Data from IMUs. (right) Reconstructed body motions. The right toe is caught on a stationary object, which interrupts its motion, forcing a quick step, followed by a recovery in mid-step on the left. The animation also shows immediate compensatory arm and trunk motion. See also the animation in supplementary material.

In the next event (Figure 4), participant #2 reported walking through heavy snow accumulation between a vehicle and a cabin, while unloading the vehicle. There were multiple events in an extended sequence, representing slipping or other faulty footsteps. Data signals show secondary foot motion during what is usually the stance phase of very low motion.

Figure 4.

Loss of balance experienced by participant #2, with voice annotation, “Activity for the last hour and half or so. Got to our cottage, about 2-3 feet snow in the driveway. Probably 100-150 foot walk from the car on the road to the house in numerous trips, probably a lot of slip and slide.” (left) Data from IMUs. (right) Reconstructed body motions. The event starts with a faulty step of the right foot, shown as an unusual slow, sideways step. It is followed by a quick compensatory left foot step. See also the animation in supplementary material.

In the final event (Figure 5), the third participant reported “Slipping and sliding on ice. Regained nay balance.” Reconstruction suggests the cause of balance loss was a slip of the right foot. This slip caused a short step and quick trunk and arm movements, along with quick corrective steps by both feet.

Figure 5.

Loss of balance experienced by participant #3, with voice annotation, “Slipping and sliding on ice. Regained my balance.” (left) Data from IMUs. (right) Reconstructed body motions. Die left foot slip is followed by several short and quick steps. The large compensatory trunk and arm response suggests a relatively severe disturbance. See also the animation in supplementary material.

IV. Discussion

This paper presents a pilot study of how a cohesive suite of wearable sensors and time-stamped event descriptions can create profiles of specific LOB events, providing richer information than other forms of home event reporting [8][9][10], while still achieving reasonable user compliance for extended durations.

The proposed approach relied not only on mounting the IMUs on typical locations such as wrist and trunk, but also on the dorsum of the feet (i.e. shoes) to reconstruct each foot’s path [15][16][17][25], The foot information helped to identify trips and stumbles, and was also instrumental for analyzing recovery from LOBs that had no obvious origin (e.g., Figure 2 and Figure 4). Sensors on the trunk and arm provided valuable additional information to estimate the approximate body pose. In some cases, a trip or slip (e.g., Figure 3 and Figure 5) initiated a chain of events culminating in upper body motion. In others, the loss of balance was quite subtle (e.g., Figure 2) and the foot motion showed how balance was recovered. Many events entailed a sequence of body motions, from foot to trunk to arm, or the opposite, that was conveyed quite naturally by animation. Creating animations with incomplete body segment data has limitations, and using multiple sensors is generally considered advantageous (e.g., [30]). But, practicality favors as few sensors as possible, especially for protocol adherence. It is hoped that such limitations in sensor wear adherence for prolonged recording periods will decrease with future improvement in sensor technology.

This method demonstrates how IMU data logging, real time verbal event reporting and human inspection may be integrated to profile individual LOBs distributed sparsely over long recording periods. This technique relies on participants to report contextual information of LOBs using voice recorders. Voice recordings were made often within a minute or two of the event, which was critical for synchronization with IMU data. In some situations where recordings were not made immediately, the synchronization process was more challenging or even impossible. To our knowledge, the detail, specificity, and nuance of the immediate recordings were more extensive than in previous reports, and, in our experience more useful than attempts to use smartphones and retrospective diaries. LOBs are difficult to describe or recall after a long passage of time [31], especially in aging populations. Contextual and other LOB and fall information is best gathered contemporaneously (see review [32]).

It is likely that many, particularly milder LOB events went unnoticed by the participant. It is therefore challenging to measure a “true” incidence of LOBs, and the degree of under-reporting remains unknown. There is also subjectivity inherent to any classification of what constitutes a loss of balance (or even a fall), for both the participant and the data evaluator. At present, some degree of subjectivity on the part of the participant or the evaluator seems unavoidable, but the process nevertheless remains useful for information on context and cause of an LOB.

The ultimate goal of long-term monitoring is to achieve automated event detection and classification, which presently perform well for relatively stereotypical events where a specific set of template motions has been recorded [22]. But in daily activity, we found many non-specific and non-stereotypical motions, such as self-generated LOBs with little or no explicit stimulus, and at times a non-stereotypical balance recovery. In the real world, the triggers for LOBs and the subsequent responses may be unique and difficult to explain without context. The disadvantage of our manual analysis process is that it was unavoidably time-intensive. Manual classification will eventually feed into future automatic event detection systems, which often need to be trained from validated proof data [19][33].

It is well recognized that environmental conditions interact with the intrinsic characteristics of individual older adults, and this interaction influences the causes of falls [34]. Participants in this study noted circumstances that challenged their balance or confidence (e.g., a poorly lit environment, or nearby ice on the ground), that are highly context specific and would be nearly impossible to identify from movement data alone. The value of this approach is that it describes enough of the body’s motion prior to, during, and after participant perception of an LOB, that it may be able to suggest a possible cause, perhaps even when the participant cannot (e.g. Figure 3). This added detail – far beyond the simple frequency of LOBs reported in past research – can move the field from classification of general fall risk to identification and reduction of specific, individual fall risk factors. For example, a cumulative assessment of the reconstructed events from subject #1 might conclude that (1) trip hazards in the household and (2) unexplained disturbances in knee function (‘knees giving out’) are significant risk factors for this individual. Recommended mitigation strategies might include instruction on eliminating trip hazards and on regular use of a balance aid for support (e.g. a cane). These hazards would not have been identified without the combination of detailed voice records and reconstructed movement animations presented for the first time in this study.

Data on a person’s functional mobility, their responses to loss of balance, and their recovery strategies, might significantly supplement the measures obtained in the clinic or laboratory with real-world mobility information. These data may be used to understand both the activity (body motions) and the environmental antecedents to an LOB, so that safer mobility strategies can be employed. In the long term, data on individuals’ LOBs may be used for customized training to reduce the occurrence of specific types of LOBs, improve intrinsic balance recovery strategies, and modify the environment to avoid falls. This approach could supplement current perturbation protocols [35] and multifactorial interventions to reduce not only falls but LOBs as well [36]. Future work should broaden these proof of concept examples to include a comprehensive evaluation of the full range of LOBs occurring during specific tasks and under specific environmental demand.

Conclusions

The proposed method provides a useful middle-ground approach to long-term wearable monitoring, using four sensors to allow an approximate reconstruction of body movement, without the wearability burden of a full-body sensor suite. Time-stamped voice recordings are a critical tool for locating rare and subtle events within a large data set, and for interpreting the meaning of the movements thus recorded. The resulting LOB profile – the reconstructed animation of each LOB plus its context in the voice recording – provides an information-rich data presentation for interpretation by a human observer. The coordinated motion of multiple segments during an LOB and subsequent recovery may enable future statistical or biomechanical analysis, and potentially patient-specific interventions.

Highlights.

• Method to reconstruct loss-of-balance (LOB) events from long-term movement monitoring

• Monitoring with wearable IMUs (feet, trunk, wrist) and a time-stamped voice recorder

• Voice descriptions record time and context of rare LOBs in a weeks-long data set

• Reconstruction and animation of body movement for interpretation by a human observer

• Description, animation and signals demonstrate LOB causes and recovery mechanisms