Clinical usefulness of augmented reality using infrared camera based real-time feedback on gait function in cerebral palsy: a case study

Abstract

[Purpose] This study investigated the effects of real-time feedback using infrared camera recognition technology-based augmented reality in gait training for children with cerebral palsy. [Subjects] Two subjects with cerebral palsy were recruited. [Methods] In this study, augmented reality based real-time feedback training was conducted for the subjects in two 30-minute sessions per week for four weeks. Spatiotemporal gait parameters were used to measure the effect of augmented reality-based real-time feedback training. [Results] Velocity, cadence, bilateral step and stride length, and functional ambulation improved after the intervention in both cases. [Conclusion] Although additional follow-up studies of the augmented reality based real-time feedback training are required, the results of this study demonstrate that it improved the gait ability of two children with cerebral palsy. These findings suggest a variety of applications of conservative therapeutic methods which require future clinical trials.

INTRODUCTION

Cerebral palsy (CP) is a non-progressive movement disorder caused by brain injury during child birth or after birth¹⁾. It leads to activity limitations, including paresthesia, cognitive impairment, dysesthesia, and behavioral disturbance²⁾. CP may affect the neuromuscular system, causing symptoms such as spasticity, contracture, muscle weakness, and loss of selective movement³⁾. In addition, children with CP never experience normal movements of the trunk, pelvis, or lower extremities against gravity⁴⁾. The characteristic motor disorders associated with CP often limit physical activity, leading to lack of motor activity, which may delay development of sensation and motor activity, sociality, etc., which may in turn limit social activity and participation⁵⁾.

Although children with spastic diplegia are able to walk, the upper limbs are raised, or the upper body overextends to compensate for the lack of control of the antigravity muscle. Improper control of the muscles around the pelvis leads to excessive anterior pelvis tilt. This forward tilt limits the hip or knee joint movements, including adduction and flexion, during gait. Impaired postural control mechanisms are a major problem affecting independent development of the activities of daily living in children with CP⁶⁾.

Treatment of children with CP is long-term, and achieving independent functional activity is very difficult. In addition, because of the shortage of skilled individuals with the ability to provide specialized care, functional activity in children with CP is often abnormal. In particular, although children with CP are able to walk independently, their gaits are not considered as normal independent walking. This abnormal gait leads to hip joint or spine deformities, as the legs asymmetrically support weight because of the bilateral asymmetry of the pelvis⁴⁾.

Active participation is emphasized during the training process. Children are offered positive rewards to encourage self-support, self-motivation, and self-satisfaction⁷⁾. If a patient is not interested in the treatment, the treatment effectiveness is reduced and the prognosis is poor⁸⁾. However, traditional methods of movement rehabilitation are composed of simple repeated movements, which do not attract a patient’s interest. Since these methods do not provide an accurate movement analysis or feedback, the effectiveness of training is further decreased because of accumulated errors due to incorrect patient movements^{9, 10)}.

Augmented feedback reportedly results in increased normal activity during the motor performance relearning process. It can provide information to patients regarding their performance of functional activities; it offers information on movement nature or quality, and it identifies both correct and incorrect components of functional activities¹¹⁾. A powerful mode of augmented feedback is the augmented reality system provided by video feedback using a computer-based simulator. Online augmented reality systems offer computer-aided instructions¹¹⁾. These systems provide real-time feedback and record the entire performance of the activity. Therefore, trainees can directly view and attempt to correct their errors¹¹⁾. However, there is a paucity of studies on the effects of real-time feedback on postural stability during exercises performed by individuals with CP.

Therefore, this study used a real-time feedback system using infrared camera recognition technology-based augmented reality for gait training of children with CP, and analyzed its impacts on gait function. The results of this study suggest the importance of the development of real-time feedback programs to improve the gait ability of children with CP.

SUBJECTS AND METHODS

The subjects were an eleven-year-old boy (height 126 cm; weight, 22 kg; Gross Motor Function Classification System (GMFCS) II level) and an eleven-year-old girl (height 112 cm; weight, 19 kg; GMFCS II level) with diplegic CP who were recruited from an elementary school for physically disabled children in Korea. The CP subjects had no history of newly developing neurological problems, musculoskeletal disorders, or botulinum toxin injections in the previous six months. The present study was approved by the Sahmyook University Institutional Review Board and the objective of the study and its requirements were explained to the subjects, and both participants provided written parental consent, in accordance with the ethical principles of the Declaration of Helsinki.

This study conducted an augmented reality-based real-time feedback training program for the two subjects over a period of two weeks, in two 30-minute sessions per week. In addition, therapists also used a mirror to provide feedback and correct the participants’ postures. The two subjects performed balance exercises intended to improve gait performance. The augmented reality-based real-time feedback consisted of two parts: the first contained eight subordinate exercise programs focused on sit-to-stand skills, while the second contained 18 exercise programs for improving standing and gait.

The augmented reality an using infrared camera-based real-time feedback training system comprised two computers (one server and one client), two cameras with infrared filters, an infrared light-emitting diode (LED) band, and head-mounted displays (HMD, SVGA resolution 800 × 600, i-Visor FX 601, Daeynag E&C Co., Seoul, Korea). The subjects wore the HMD to provide real-time feedback during gait training¹¹⁾. The HMD is an augmented reality system that blends virtual movement with real movement in real-time^{11, 12)}. Each subject with CP received this gait training at his or her physical therapy clinic while wearing comfortable clothes.

The augmented reality system provided guidance based on virtual movement superimposed on the actual movement in order to help the subjects perform more normal activities during gait training. The actual movements were performed by the subjects, while the virtual movement was performed by a young boy performing the standard program exercise. The subjects watched the young boy perform the standard exercise and copied the movement using infrared camera recognition technology. The subjects were tracked and calibrated via a video approach during the exercise and were subsequently instructed to modify the movement of specific body parts based on the locations of infrared LED markers during the virtual movement. Graphics and vision-based camera recognition, visual representation technology for emotional feedback, audio-visual models and a database of cognitive information, surrounding situational awareness, and response-action processing technologies were used to supplement the lack of sensory, motor, and cognitive function associated with the augmented reality environment. In addition, real-time motion recognition, based on the locations of infrared LEDs, was used to assess the speed and accuracy of the subjects as they imitated the pre-defined model behaviors. Therapists suggested appropriate exercise programs after evaluating the subjects (Fig. 1). The subjects wore the HMD and received training by following the suggested motions in front of a camera installed on a computer. They were directed to repeat the training tasks or to learn advanced programs based on their performance as monitored by the computer-based system. Based on the CP severity and effects, a variety of appropriate treatment environments were offered to the subjects.

Fig. 1.

Schematic diagram of the augmented reality using an infrared camera for real-time feedback

To measure the gait function, we used GAITRite (GAITRite, CIR system Inc., USA, 2008), a commercially available gait analysis system that instantaneously measures velocity as well as step cadence, step length, stride length, and the functional ambulation performance score (FAPS). The FAPS is integrated within the GAITRite walkway, which is considered to be the gold standard for spatiotemporal gait parameter analysis. The FAPS is commonly used for clinical evaluations. Three test trials were performed after a practice trial in this study. The mean of the test trials was calculated.

SPSS version 17.0 for Windows was used for all analyses. The means of spatiotemporal gait parameters were used for analysis.

RESULTS

The gait velocity of case A increased from 68.3 cm/s at pre-test to 99.5 cm/s at post-test, the cadence increased from 94.0 to 94.7 steps/min, the right-side step length from 41.33 to 48.33 cm, the right-side stride length from 86.08 to 102.58 cm, and the FAPS from a score of 85 to 89.

The gait velocity of case B increased from 56.8 cm/s to 60.6 cm/s, the cadence from 89.3 to 91.7 steps/min, the right-side step length from 34.52 to 37.52 cm, the right-side stride length from 71.39 to 77.1 cm, and the FAPS a score from 91 to 93 (Table 1).

Table 1. Pre-test and post-test spatiotemporal parameters.

Parameter		Case 1		Case 2
		Pre-test	Post-test	Pre-test	Post-test
Velocity (cm/s)		68.3	99.5	56.8	60.6
Cadence (step/min)		94.0	94.7	89.3	91.7
Step length (cm)	Right side	41.3	48.33	34.52	37.52
	Left side	38.19	51.44	36.61	39.20
Stride length (cm)	Right side	86.08	102.58	71.39	77.1
	Left side	95.92	96.61	72.03	77.03
FAPS (score)		85	89	91	93

Values are means; FAPS: functional ambulation performance score

DISCUSSION

Current neurodevelopmental treatment techniques include progressive muscle strength exercise, feedback training, child-centered task-oriented training⁴⁾, and electrical stimulation to rehabilitate neurological damage. However, virtual reality offering three-dimensional images has recently been used to treat and evaluate patients with brain damage; these virtual reality treatments use video games, virtual reality-based ankle exercise¹³⁾, and brain-computer interface-based functional electrical stimulation¹⁴⁾. Virtual reality training programs for eye-hand coordination¹⁵⁾, balance¹⁶⁾, gait speed¹⁷⁾, therapy games¹⁸⁾, and home-based virtual reality intervention¹⁹⁾ programs have been used for patients with CP. Virtual reality interventions improve patient functions, and are highly reliable and valid assessment tools²⁰⁾. Virtual reality is a particularly effective intervention for cognitive impairment and motor dysfunction, because the level of difficulty of the virtual environment and task performance can be easily modified²¹⁾.

Studies of the effect of virtual reality training exercises for rehabilitation of CP are scarce²²⁾. However, the results of previous studies of virtual reality training suggest that patients find this technology interesting and engaging, suggesting its usefulness as a tool to motivate patients^{22, 23)}. Therefore, this study was conducted with augmented reality-based real-time feedback training to examine its effect on the improvement of the gait ability of patients with CP. The server computer controlled the training program for CP, infrared camera monitoring, and real-time motion recognition using infrared LEDs for the assessment of subject speed and accuracy, while the client computer received video from the server computer and presented an edited version to the children with CP via the HMD. During this process, the children with CP were able to compare their motions with the examples provided by the augmented reality-based system.

Reactions in standing postures are diverse, from simple single synaptic reflexes to activation of movement strategies. Typically, body deformation due to a disability prevents patients from maintaining good posture or weight support in comfortable standing positions. A variety of therapeutic treatments are used to address this issue, including rest feedback methods such as responses and specific stimulation to improve postural control, feed-forward methods to assist proper postural control by organizing muscle coordination and timing, separating movement patterns of both lower extremities with characteristics of consistent intensity, and the trunk muscle contraction required for balance while standing. In particular, the results of this study show that HMD augmented reality offers trainees the ability to continue to modify their incorrect movement, which is one of the best methods for stimulating feed-forward loops. In this study, real-time feedback using infrared camera recognition technology-based augmented reality during gait training of children with CP resulted in improved gait function, as evidenced by increased gait velocity, cadence, step length, stride length, and functional ambulation.

Real-time feedback was provided to two children with CP wearing an LED band via infrared LED camera recognition technology that detected their movements. Infrared rays can be used to detect motion regardless of lighting conditions, but are influenced by the number of infrared LEDs; however, increasing the number of infrared LEDs accelerates battery depletion. This study used four infrared LEDs to detect the subjects’ motion.

The results of this study show that an augmented virtual reality exercise program improved the ambulation ability of two subjects with CP. These findings suggest that this exercise program should be recommended for the treatment of CP as well as for patients with neurological problems, particularly those with stroke. Effective programs should be continuously studied and disseminated in order to allow patients to return to their social communities.