Utility of a Three-Dimensional Interactive Augmented Reality Program for Balance and Mobility Rehabilitation in the Elderly: A Feasibility Study

Abstract

Objective

To improve lower extremity function and balance in elderly persons, we developed a novel, three-dimensional interactive augmented reality system (3D ARS). In this feasibility study, we assessed clinical and kinematic improvements, user participation, and the side effects of our system.

Methods

Eighteen participants (age, 56-76 years) capable of walking independently and standing on one leg were recruited. The participants received 3D ARS training during 10 sessions (30-minute duration each) for 4 weeks. Berg Balance Scale (BBS) and the Timed Up and Go (TUG) scores were obtained before and after the exercises. Outcome performance variables, including response time and success rate, and kinematic variables, such as hip and knee joint angle, were evaluated after each session.

Results

Participants exhibited significant clinical improvements in lower extremity balance and mobility following the intervention, as shown by improved BBS and TUG scores (p<0.001). Consistent kinematic improvements in the maximum joint angles of the hip and knee were observed across sessions. Outcome performance variables, such as success rate and response time, improved gradually across sessions, for each exercise. The level of participant interest also increased across sessions (p<0.001). All participants completed the program without experiencing any adverse effects.

Conclusion

Substantial clinical and kinematic improvements were observed after applying a novel 3D ARS training program, suggesting that this system can enhance lower extremity function and facilitate assessments of lower extremity kinematic capacity.

INTRODUCTION

Falls and fall-related injuries are common and serious medical problems among elderly people. Impaired motor performance in older adults is frequently characterized by motor retardation and decreased muscular power and balance. These deficits are primarily responsible for falls. More than 33% of community-dwelling individuals aged >65 years fall at least once per year; 50% of such individuals will experience recurrent falls [1]. The frequency of falling increases up to 60% with age; three-quarters of all deaths caused by falls in the United States occur in those ≥65 years [1,2].

Many factors are involved in falls and fall-related injuries but a balance disorder is the most important factor in older adults. The most effective method for ameliorating fall risk is physical exercise targeting lower extremity strength, balance, and proprioception [3]. Additionally, good cognitive function and attention are necessary to maintain proper balance and gait [4]. Specific exercises to improve balance include novel gaming systems using interactive computer-generated feedback environments and goal-directed tasks [5,6].

Virtual reality (VR) applications are relatively novel and potentially useful rehabilitation techniques. Moreover, rapid advances in interface and augmented reality (AR) technologies and various sensorimotor feedback techniques are being made [7]. Virtual and augmented environments are effective for rehabilitation training, as they apply motor learning principles [8]. Virtual environments immerse the user within an entirely simulated space, whereas augmented environments provide an interactive experience anchored in physical reality, thereby allowing knowledge gained in the therapeutic setting to be used in daily life [9].

In addition, high-resolution motion capture can provide clinically valuable kinematic information concerning participant performance. A computational kinematic evaluation helps supplement data from existing clinical scales by providing reliable and repeatable quantitative measures of movement. This type of analysis allows for continuous measurement of many aspects of movement (e.g., velocity and joint angles). Linking motion-sensing with visual and audio feedback creates an immersive interactive experience that provides detailed information about performance and facilitates active engagement and sensorimotor learning [10].

Cognition function, proprioceptive input during exercise, and learning ability decrease gradually with increasing age; thus, an AR-based exercise program could be useful for older adults. Such a system would provide intuitive external feedback on movement, thereby augmenting proprioception arising from intrinsic sensory organs whose function may have been compromised by aging, while simultaneously encouraging sensory-motor integration [11]. Feedback also offers guidance, motivation, and encouragement and may improve movement quality and confidence. Lower extremity exercises using an ARS demonstrably enhance balance in elderly persons; however, the majority of studies have targeted patients with neurological deficits [8,12]. The utility of this novel system for balance rehabilitation in older adults has not been evaluated. Therefore, there is a lack of evidence regarding its efficacy compared with conventional balance exercises or that of commercial devices using AR interactive videogames to improve lower extremity function.

We developed a three-dimensional interactive augmented reality system (3D ARS) to improve lower extremity function and balance in elderly persons. The 3D ARS integrates traditional rehabilitation and motor learning theories with high-resolution motion capture and sensing technologies, 'smart' physical objects, and interactive computer graphics and sound. The system allows for kinematic measurements, derived from motion capture, to help assess the individual components of movement. We hypothesized that this system will enhance lower extremity function. In this feasibility study, we evaluated the adverse effects and documented any user issues.

MATERIALS AND METHODS

Participants

Eighteen community-dwelling older adults were recruited. All participants provided informed consent and were cognizant of the tasks before the study commenced. The inclusion criteria were as follows: 1) age 55-80 years; 2) Mini-Mental State Examination (MMSE) score ≥25 [13]; 3) ability to understand the nature of the study and provide informed consent; and 4) capable of independent in ambulatory functions, with or without an assistive device (cane or walker), and the ability to stand on one leg. Exclusion criteria were any medical condition or disability that prevented participation in an exercise program, current treatment for a medical disease, recent fracture, uncontrolled diabetes or seizures, cardiovascular problems prohibiting exercise, and neurological disorders. This study protocol was approved by the local Institutional Review Board at Eulji Hospital.

Development of the 3D ARS

The 3D ARS was developed by clinicians, physical and occupational therapists, and biomedical and software engineers. The software (Metasio Asia Inc., Korea) operates in a Windows 7 environment, employs an AR setting, and motion-tracking kinetic sensor.

Setup and hardware

The 3D ARS utilized a Kinect Xbox 360 (Microsoft Inc., Redmond, WA, USA) to detect and track subject body movements and interactions with virtual objects. The Kinect sensor generates a 3D depth map, such that objects in front of the sensor can be detected in 3D space. Individuals positioned in front of the sensor are detectable as 3D objects; the position and angle of the joints in 3D space can be assessed using a skeleton image extrapolated from the user's depth image. Joint angles are subsequently accounted for during game-playing sessions.

The augmented environment was generated by superimposing video of real objects onto images of virtual objects rendered by a personal computer. Subjects interacted with virtual objects displayed on a screen in front of them and were able to see their appearance (i.e., gait and posture, etc.) in real time.

Task types

Three tasks were designed to elicit specific motions with specific joints: the 'balloon game', the 'cave game', and the 'rhythm game' (Fig. 1). Each game focused on a specific exercise. For example, the balloon game encouraged hip flexion and internal and external hip rotation to touch a falling balloon; the cave game encouraged flexion and extension of the knees to avoid obstacles in a cave; and the rhythm game was designed to enhance the ability to stand on one leg by stepping on specific locations appearing on the floor.

Fig. 1. Participants engaged with a three-dimensional, interactive, augmented reality rehabilitation system, comprised of the balloon game (A), the cave game (B, C), and the rhythm game (D).

Balloon game

The balloon game was designed to train the hip joint. Balloons fall from the top to the bottom of the screen, and patients touch and burst the balloons with their foot by performing flexion, extension, and external and internal rotation of the hip joint. The balloons fell in random positions (five levels in total). The balloons fell more rapidly and frequently as the level of difficulty increased, thereby necessitating more rapid but balanced movements. The interval between balloon appearances during the experiment was adjusted to each individual's ability; the average interval was 3 seconds. Falling speed varied from 3.5 to 7 seconds to reach the floor; speed was assigned randomly but evenly distributed among possible speeds. The subjects could burst approximately 200 balloons during the 10-minute trial.

Cave game

The objective of the cave game is to avoid collisions with stalactites emanating from the ceiling of a cave by flexing the knee. A cave image moves from right to left, such that users feel as if they are proceeding through a cave. As difficulty level increases, the stalactites become longer, and the cave moves more quickly. In the present experiment, the stalactites appeared for 5 seconds, and a new one was generated 1 second after each passed. Thus, subjects encountered approximately 100 stalactites during the 10-minute trial.

Rhythm game

The rhythm game involves eight virtual plates situated around the player. Patients touch the plates by moving their feet. The plates appear in seven locations: front-left, front, front-right, left, right, back-left, and back-right. By encouraging users to move their feet, the game enhances standing stability. As difficulty level increases, the interval between and the duration of the plate's appearance shortens. For example, a subject would need to move their feet within 3 seconds after a plate appeared, and the next plate would appear 1.5 seconds after the subject stepped on the first plate, which was visible for 3 seconds. Thus, a subject would step on approximately 200 plates during the 10-minute trial.

Exercise dynamics

Each game highlighted different aspects of user performance. Response time (s) and success rate were outcome measures used for all three games. Response time (s) refers to the time taken to complete a task in a given game session; success rate refers to the percentage (%) of tasks successfully completed during each game session. Mean response time and success rate values were analyzed offline after completing the intervention.

Several movement parameters for the joint of interest in each game were also recorded during the exercises: the angle of hip flexion and external and internal hip rotation during the balloon game, and the knee flexion angle in the cave and rhythm games (Fig. 2).

Fig. 2. The angle of each joint (curved arrow) was calculated based on joint position using the 3D kinetic sensor. (A) External hip rotation angle, (B) internal hip rotation angle (angle of a vector from the knee to the ankle, with a vertical reference vector in the coronal plane), (C) hip extension and flexion angle, and (D) knee flexion angle (angle of a vector from the hip to the knee, with a vertical reference vector in the sagittal plane).

Training and clinical evaluation

Participants received 10 sessions of 3D ARS training (30 min/session) over the course of 4 weeks, and the training level was adjusted to each individual's motor abilities. All participants were subject to a clinical evaluation prior to and following the intervention, using the Berg Balance Scale (BBS) and the Timed Up and Go (TUG) test. A physical therapist, with 10 years of experience, administered the tests. Participants were briefed about the nature of the intervention prior to the evaluation. The BBS is widely used to assess static and dynamic balance capabilities [14]: scores range between 0 and 56, with higher scores indicating superior balance. The TUG measures the time required to perform the following actions: stand up from a chair; walk 3 m at a normal pace; and turn around, walk backwards, and sit down [15]. The TUG is commonly used to examine functional mobility and balance [16].

Evaluation of side effects and use issues during 3D ARS training

The Pittsburgh Rehabilitation Participation Scale (PRPS) was used to evaluate the interest and motivation of participants following each 3D ARS session [17]. PRPS scores range from 1 to 6, as follows: 1, patient refused to participate in any exercises; 2, patient did not participate in at least half of the exercises; 3, patient participated in most or all of the exercises, but did not exhibit maximal effort or complete the majority of the exercises, or required significant encouragement to complete the exercises; 4, patient participated in all exercises, exhibited a high degree of effort, and completed most but not all exercises and passively followed directions; 5, patient participated in and completed all exercises, demonstrated maximal effort, but passively followed directions; 6, patient participated in and completed all exercises, demonstrated maximal effort, and took an active interest in the exercises. Additionally, we assessed any side effects, such as dizziness, headache, falling, and joint pain, which might be related to participation in the 3D ARS training.

Statistical analysis

Data are presented as mean±standard deviation (SD), median, or range. The normality of the continuous variables was assessed using the Kolmogorov-Smirnov test. Normally distributed variables are presented as mean±SD. We used the paired t-test to assess changes in clinical values before and after sessions. Changes in the performance scale and kinematic values for each session were assessed using linear regression analysis; rate of change is presented as regression coefficients (B) and the standard error (SE). p<0.05 was considered significant; p<0.2 was considered indicative of a trend [18]. All data were analyzed using SPSS ver. 18 software (SPSS Inc., Chicago, IL, USA).

RESULTS

Participant characteristics

The characteristics of the 18 participants were: age, 64.70±7.27 years (range, 56-76 years); height, 1.64±0.08 m; body mass index, 24.09±2.77 kg/m²; MMSE score, 29.06±1.00; BBS score, 54.56±1.54; and TUG score, 7.85±0.72 (Table 1).

Table 1. Demographic and baseline characteristics of the participants.

BMI, body mass index; MMSE, Mini-Mental State Examination; BBS, Berg Balance Scale; TUG, Timed Up and Go test.

Clinical improvement following 3D ARS training

Participants exhibited improved balance and mobility following 4 weeks of 3D ARS training, as indicated by significantly higher BBS score (baseline, 54.56±1.54; post-intervention, 55.50±0.92; p<0.001) and TUG score (baseline, 7.85±0.72; post-intervention, 7.35±0.67, p<0.001) (Table 2).

Table 2. Changes in balance and gait parameters before and after 3D ARS training.

Values are presented as mean±standard deviation.

3D ARS, three-dimensional interactive augmented reality system; BBS, Berg Balance Scale; TUG, Timed Up and Go test.

Improved performance during 3D ARS training

Mean response time decreased significantly across sessions during the balloon game (B=-0.138, SE=0.030, p=0.002), cave game (B=-0.410, SE=0.120, p=0.011), and the rhythm game (B=-0.037, SE=0.009, p=0.003). The success rate for the balloon game increased significantly across training sessions (B=1.140, SE=0.145, p<0.001). A trend toward an increased success rate (B=0.083, SE=0.038, p=0.059) was observed during the cave game. Success rate increased significantly across sessions of the rhythm game (B=0.155, SE=0.213, p=0.001) (Fig. 3).

Fig. 3. Mean success rate (%) and response time (s) to stimuli for each trial in each game improved as sessions progressed: (A) ball game, (B) cave game, and (C) rhythm game. Error bars indicate standard error.

Kinematic improvements during 3D ARS training

The hip flexion task used during the balloon game was associated with the highest rate of success, followed by the external rotation and internal rotation tasks (79.35%±14.95%, 78.51%±18.92%, and 64.82%±26.34%, respectively). The average degree of hip flexion across all sessions was 26.07°±10.82°; for external rotation 24.66°±10.23° and for internal rotation 25.04°±10.33°. In the linear regression analysis, maximum hip external rotation improved significantly across sessions (B=0.644, SE=0.212, p=0.016). A trend toward improved maximum knee flexion angle was observed during the cave game (B=0.530, SE=0.230, p=0.050) (Fig. 3, Table 3). The mean knee flexion angle decreased significantly during the rhythm game (B=-0.179, SE=0.044, p=0.003).

Table 3. Changes in kinematic and performance outcome variables during 3D ARS sessions: regression analysis.

3D ARS, three-dimensional interactive augmented reality system; B, regression coefficient; SE, standard error.

Evaluation of use issues and side effects

18 participants completed 10 3D ARS sessions without experiencing any adverse effects, although one subject exited the trial for private reasons. PRPS scores increased significantly during training across sessions (B=0.208, SE=0.520, p<0.001) (Fig. 4). By the end of the intervention, all participants scored 6 on the PRPS, i.e., they participated in and completed all exercises, and demonstrated maximum effort and an active interest.

Fig. 4. Maximum hip external rotation during the balloon game (A) and maximum knee flexion angle during the cave game (B), increased following training. Mean success rate (%) per target in the rhythm game (C) increased as sessions progressed. (D) The Pittsburgh Rehabilitation Participation Scale (PRPS) scores after every session also increased. Error bars indicate standard errors.

DISCUSSION

Ten sessions of 3D ARS training delivered across 4 weeks was associated with significant improvements in lower extremity function and balance in an elderly cohort, as indicated by improvements in the BBS and TUG scores, as well as kinematics (e.g., maximum joint angle of the hip and knee). Furthermore, performance outcome variables, such as success rate and response time of each exercise, improved steadily across sessions. Furthermore, participants exhibited interest in and completed all of the exercises without experiencing adverse effects.

We analyzed maximum joint angles across sessions because they could reflect improvement in performance during the game. Maximum external hip rotation and knee flexion angles increased significantly or showed a trend toward improvement across sessions. However, improvements in maximum flexion and internal rotation of the hip were not significant. The mean hip flexion angle had the highest score, and the success rate was the highest among the tasks, so we assumed that there was no improvement in maximum hip flexion angle across sessions. In contrast, the mean degree of hip internal rotation and success rate were the lowest among the tasks; thus, we conclude that there was no significant improvement in this domain during these relatively short exercise sessions. Mean knee flexion angle decreased significantly, as reflected by improved ability to stand on one leg during the rhythm game.

Only group- and home-based exercise programs have been associated with a reduced risk of falling [19]. Individualized progressive resistance strength training, dual-task motoric balance training, visual biofeedback training, and dynamic balance exercise training, undertaken on fixed and compliant sponge surfaces, improve balance and gait in older adults [6,20,21,22]. In addition, 10 weeks of commercial VR program training, using videogame software (PlayStation 2; Sony, Tokyo, Japan) improves balance, strength, gait, and fall rates in elderly adults with diabetes [23]. Similarly, healthy elderly participants engaged with a 30-minute exercise session using the 'Wii Fit' software package (Nintendo, Kyoto, Japan) three times a week for 8 weeks, exhibited improved balance [24]. However, there is limited evidence that specific training (i.e., for gait, balance and coordination, strengthening, and 3D and multiple-exercise regimes) effectively improves balance in older people compared with conventional exercise [25].

Applications of VR technology to balance rehabilitation have attracted attention in the field of neurological rehabilitation [26]. VR involves computer-generated interactive environments, in which participants experience imaginary objects and scenarios as real [27]. Neurologically impaired subjects have reported that activities undertaken in these virtual environments, such as walking and manipulating objects, feel authentic [28]. There is increasing evidence that VR training may benefit mobility and balance of individuals with neurological disorders, thereby reducing the risk of falling, but limited evidence suggests that VR improves balance to a greater extent than conventional physiotherapy [29].

However, VR applications have significant potential for balance rehabilitation because interfaces, AR technologies, and sensorimotor feedback techniques are improving rapidly [30]. In the present study, we employed software operating in conjunction with a motion-tracking kinetic sensor. In contrast to the Nintendo Wii and Sony PlayStation systems, which generate complete synthetic environments navigable by an avatar, ARS enhance the non-synthetic environment by introducing synthetic elements to the user's perception. VR replaces the existing physical environment with a virtual one, whereas AR uses virtual elements to augment the existing environment [31]. In contrast to VR, participants 'maintain a sense of presence' in the non-synthetic world because in AR they do not 'depart' the physical space that they occupy [32]. The feeling of 'being there' underpins the potential of AR for rehabilitation; several studies have reported that only actions completed in a non-synthetic environment activate the visuospatial network of the brain [33,34].

AR may also improve performance in older, neurologically impaired patients. The AR-based 'Otago Exercise' is associated with significant improvements in BBS scores and gait and a reduced risk of fall [35]. ARS training involving a first-person view of virtual lower limbs improves balance and strength in patients with an incomplete spinal cord injury [36]. Use of VR or AR exercise systems has several advantages. Interactive virtual environments can augment postural control in older adults by stimulating the sensory processes responsible for maintaining balance and orientation. Virtual augmented environments also can specifically target motor learning by selectively activating the motor areas of the brain [37].

Furthermore, the high-resolution motion capture technology of our 3D ARS provides valuable kinematic information on patient performance and creates a motivating, interactive experience that facilitates active engagement and sensorimotor learning by linking motion-sensing with visual and audio feedback. [38], The Kinect sensor does not require attachment of any other device to the user's body to track objects in 3D space, and the sensor recognizes motion effectively. In the present study, the maximum hip and knee joint angles increased steadily across each session, suggesting that the sensor system was reliable. However, there is also a limitation associated with use of Kinect, namely that objects must be located within a specific range to be detectable by the sensor. Kinect recognizes users standing at distances of 0.8-4 m, but a range of 1.2-3.5 m is optimal, to allow for extension and use of the arms. The Kinect sensor can also be affected by environmental lighting conditions because it uses an infrared camera. Concurrent use of several Kinect sensors or a specific lens to widen its range could counter the limitations.

In the present study, improvements in balance were small and did not meet the criterion of clinical significance achieved in previous studies reporting significant differences based on large SD values across patients [39,40]. The healthy aged population in our study showed relatively high clinical balance scores and small SD values, so the small change in clinical values reported here may be meaningful. Furthermore, our training period was relatively short, and the sample size was small. Further trials that examine the usefulness of the 3D ARS in a large study are required to compare this approach with conventional balance exercise programs in a long-term training program.

The present feasibility study demonstrated that intensive 3D ARS training across 10 discrete sessions (30 min/session) over a 4-week period is a safe, well-tolerated, and efficacious method of improving balance and mobility in healthy elderly people, principally by enhancing lower extremity function. Moreover, the system allows for a detailed kinematic assessment. A future randomized controlled study is needed.