Modifying and evaluating efficacy of interactive computerized program using motion tracking technology to improve unilateral neglect in patients with chronic stroke

Abstract

Background:

To modify and evaluate the efficacy of a computerized visual perception rehabilitation program using interactive motion tracking technology with unilateral neglect after chronic stroke.

Methods:

Study design is single-blinded (analyst-blinded) controlled prospective clinical trial. Subjects are 16 patients with chronic stroke and unilateral neglect for over 6 months and 19 healthy volunteers. We modified our previous program to 9 tasks with built-in scoring system, and the subjects performed 3 sessions per week, 30 minutes per session for 4 weeks.

Results:

Scores for the Modified Barthel Index (MBI), Mini-Mental State Examination (MMSE), Motor-free Visual Perception Test (MVPT), Line bisection test, Star cancellation test, Forward Digit Test, and Backward Digit Test showed significant improvement at the end of the sessions in the patient group. By comparing the parameters of built-in scoring system of each task among the control group, the first session of training in the patient group, and the last session of training in the patient group, we categorized the parameters for optional measurement to determine the effect of training or to be a candidate for evaluative use.

Conclusions:

Our modified computerized visual perception rehabilitation program using improved unilateral neglect in patients post-stroke. Built-in scoring system in this program was helpful to assess availability of it more objectively.

1. Introduction

Unilateral neglect syndrome after stroke is known to be very common with a reported range from 20% to 82%, and negatively influences the activities in daily living (ADL).^[1] To improve the unilateral neglect syndrome after stroke, several treatment interventions such as computerized cognitive programs, prism adaptation, mirror therapy, somatosensory electrostimulation, optokinetic stimulation, virtual reality (VR), transcutaneous electrical nerve stimulation, and pharmacological therapies have been actively and effectively investigated.^[2,3]

We developed a computerized visual perception rehabilitation program with an interactive computer interface using motion tracking technology in 2009.^[4] Our program was one of the computerized cognitive rehabilitation programs, and it was designed to give direct visual feedback of their own hands to the subjects by using interactive motion tracking technology. Moreover, this program was more user-friendly because it is performed by subjects’ own hand, not by the input devices. This program was composed of 12 computerized tasks categorized in 4 parts designed to improve visual perceptual function. These tasks were designed to train visual recognition, visual differential reaction, visual tracking and targeting, and visual spatial cognition and motor functions, and showed improvement of unilateral neglect after stroke which was comparable to an existing computer-based rehabilitation program.

However, our previous study was performed in the early phase of development of the program, which was composed of relatively simple tasks. Moreover, the subjects were in the acute stage of stroke, which made it difficult to differentiate between the effect of the program and spontaneous improvement. We thought that additional results in patients with chronic stroke would be helpful to know the advantage of the developed program. In addition, we compared the efficacy of our program with the commercialized program (PSS CogReh) in the previous study; however, we were unable to show objective improvement in the program itself. Development of a built-in scoring system is required to assess the change of the visual perception more objectively.

The purpose of this study was to modify the previous computerized visual perception rehabilitation program to have a built-in objective scoring system, and to apply this program to the patients with visual perceptional impairment of >6 months.

2. Methods

2.1. Subjects and controls

The subjects were patients with visual perceptional impairment of >6 months after right hemispheric stroke. They were recruited from the outpatient unit of Chung-Ang University Hospital in Seoul, Korea. Inclusion criteria were: right hemisphere stroke, left hemiplegia with normal motor function in right side; after 6 months since the onset of stroke, confirmed visual perceptional impairment through Motor-free Visual Perception Test (MVPT-3, standard score <109); and Mini-Mental State Examination (MMSE) score of >18 points. Exclusion criteria were: visual impairment from diabetic retinopathy or senile cataract; and hearing difficulty or cranial nerve dysfunction. Eighteen patients who volunteered for the study were evaluated for eligibility and 2 patients were excluded because 1 was unable to complete during the initial evaluation, and 1 had diabetic retinopathy. Of the 16 remaining patients, 8 patients had hemiplegia from hemorrhage, and 8 patients from infarct. The mean age of the patients was 63.7 years. There was no statistically significant difference in MVPT, MMSE, and Modified Barthel Index (MBI) score between the hemorrhagic stroke patients and ischemic stroke patients.

The control group was comprised of 19 healthy volunteers with similar ages as the subjects (age and sex matched).

2.2. Programs

The technology of the program has the same CAMSHIFT (continuously adaptive mean shift) algorithm as used in our previous study.^[7] In short, the camera detects and follows the movement of a hand and displays it on the monitor. To localize the movement, we used a small blue cap covering one finger. We modified our previous programs to 9 tasks in the computerized visual perception rehabilitation program including “Click,” “Frying pan,” “Color,” “Bubble,” “Shape,” “Figure,” “Assembly,” “Memory,” and “Track” (Fig. 1). When the participant is fitted with the blue cap on one finger and raises his right hand in the air, the camera captures and displays the location of the cap on a monitor as a blue hand, and the system starts to follow the movement. The finger could be chosen by the participants’ convenience, and second fingers were usually chosen. The participants were then asked to perform the following 9 tasks shown on the computer screen.

Figure 1.

Nine tasks. (A) In the “Click” task, the subject is asked to move the finger to touch the orange ball presented, and then to touch the cup to complete the performance. (B) In the “Frying pan” task, the subject is asked to move the finger presented as a frying pan to catch only eggs, trying not to catch the chicks falling from the upper side of the screen. (C) In the “Color” task, the subject is asked to choose the same colored box in the lower part of the screen according to the color of the box shown on the upper side of the screen. (D) In the “Bubble” task, the subject is asked to choose the figure from the 3 figures that was thought to be obscured by the bubble on the left part of the screen. (E) In the “Shape” task, the subject is asked to touch the same figures on the screen according to the specific figure shown inside a basket at the bottom of the screen. (F) In the “Figure” task, the subject is asked to touch the figure with the same shape as the red figure on the left top of the screen, and touch the basket to complete the task. (G) In the “Assembly” task, the subject is asked to remember the arrangement of the 2 figures presented first, and then select the figures to reproduce the arrangement. (H) In the “Memory” task, the subject is asked to remember the order of the figures to be touched by the blue hand, and then to reproduce the order of touch. (I) In the “Track” task, the subject is asked to follow the direction presented by an arrow to touch the figures on the screen.

We designed these 9 tasks and tested various levels of difficulties in each task in healthy volunteers to determine the proper level to be performed in the study. Our goal of the level of difficulty was intended to be easy for healthy volunteers and moderately difficult for the patients with neglect. We also adjusted the level of difficulty not too hard for the patients, because we wanted improvement in each task at the end of the training session. We used the same level of difficulty for the all participants. We determined the scoring system, numbers of items in each task, and number of repetitions (Table 1). For each task, we identified 3 outcome parameters in this study: performance scores, performing time, and errors. Performance scores were calculated by assigning a point for each successfully completed item in each task. Total scores for each task varied, as shown in Table 1. If the test was repeated more than once, the mean value of the tests performed was used. To calculate the performing time spent to complete each task, (mean) total performing time was divided by the number of items. Tasks are described below:

Table 1.

Test configurations.

(1) “Click” (Fig. 1A): The screen showed a cup in the lower part of the monitor. The participant was asked to concentrate on the screen until an orange ball was shown. When the ball appeared, the participant was required to move the finger to touch the ball, and then touch the cup to complete the performance. “Click” program is used to evaluate the visual recognition.

(2) “Frying pan” (Fig. 1B): The participant's finger was displayed as a pan. Subsequently, eggs and chicks fall from the top of the monitor. A participant was instructed to move the finger to catch only eggs, trying not to catch the chicks. Errors were computed as the number of chick caught. “Frying pan” evaluates visual recognition.

(3) “Color” (Fig. 1C): Four color boxes are shown in the lower part of the monitor. The participant was asked to concentrate on the upper part of the monitor until it showed a new color box. On seeing the color of the new box, the participant was asked to choose the box from the boxes shown in the lower part that matched the color of the upper part of the screen. “Color” evaluates basic visual perception.

(4) “Bubble” (Fig. 1D): The left part of the screen showed a figure partially obscured by the bubble. Subsequently, 3 different unobscured complete figures were shown on the right side of the screen. The participant was asked to move the finger to choose the figure from among the 3 figures that is the same as the figure on the left side. “Bubble” evaluates basic visual perception.

(5) “Shape” (Fig. 1E): A figure inside a basket was presented on the bottom of the screen. Several figures were shown on the monitor, outside the basket. The participant was asked to confirm the shape of each figure inside the basket and touch the figures outside the basket that had the same shape as a figure inside the basket. After matching all the figures, the participant was required to touch the basket to complete the session. Errors were the count of numbers of incorrect matches of figures from the outside to the inside of the basket. “Shape” evaluates basic visual perception.

(6) “Figure” (Fig. 1F): A red figure was presented on the left top and a basket on the left bottom of the screen. Four different figures were shown on the monitor, and the participant was asked to touch the figure with the same shape as the figure on the left top, and touch the basket to complete. “Figure” evaluates basic visual perception.

(7) “Assembly” (Fig. 1G): First, the participant was presented with 2 boxes with a certain shape of figures in the middle of the screen, and asked to remember the arrangement of the 2 boxes. Then the monitor was cleared, and 2 boxes with figures with the same shapes with those shown at first were shown on the right side of the screen with random arrangement. If the participant touched to select these boxes in turn, empty boxes shown in the middle of the screen were filled with the selected box. The participant was required to reproduce the original arrangement. “Assembly” evaluates complex visual perception.

(8) “Memory” (Fig. 1H): First, the participant was asked to concentrate on and remember the order of movement (trajectory) of the blue hand on the screen. Then, the participant was asked to touch the figures to reproduce the trajectory shown before. To complete the session, the participant was asked to touch the basket in the middle of the screen. “Memory” evaluates visual memory.

(9) “Track” (Fig. 1I): When participant was ready, the camera captured and displayed the finger in the air. Then an arrow was displayed to a certain direction and the participant was asked to follow the direction and touch the figure directed. Several arrows were displayed continuously. “Track” evaluates complex visual perception.

2.3. Study design

This study is single-blinded (analyst-blinded) controlled prospective clinical trial.

2.4. Intervention and outcome measurements

All participants were assigned to complete the programs under the supervision of an occupational therapist, and performed 3 sessions per week for 30 minutes per session for 4 weeks.

For each session, parameters of the program such as performing time, performance score, and errors were recorded and tracked over time.

For the functional evaluation, MBI, MMSE, Motor-free Visual Perception Test (MVPT, MVPTS, and motor-free visual perception test – revised [MVPTR]), Line bisection test, Star cancellation test, Lowenstein Occupational Therapy cognitive Assessment (LOTCA), Forward Digit Test (Digit F), and Backward Digit Test (Digit B) were assessed at the beginning and at the end of the session to be compared.

2.5. Statistical analysis

Data obtained at the first and last session were compared using Wilcoxon signed-rank test. A P-value <.05 was considered significant. Data analyst was blinded to the data.

2.6. Ethics statement

The study protocol was approved by the Institutional Review Board of Chung-Ang University Hospital, IRB No. C2009053 (237). Informed consent was confirmed by the board.

3. Results

The results of functional evaluations are shown in Table 2. These tests were performed before starting the first session of the program and they were repeated at the end of the last session which was 4 weeks after the first session. Scores for the MBI, MMSE, MVPT (MVPTS and MVPTR), Line bisection test, Star cancellation test, LOTCA, Digit F, and Digit B showed significant improvement after program completion (P < .05) (Table 2).

Table 2.

Comparison of the results of functional tests of the patients group performed between before starting the first session and after end of the last session of the program.

The results obtained from each individual task of the program are shown in Table 3. Every task had the parameter of performance scores and performing time, except for the task of “Frying pan” which included the parameter of performance scores and errors, and “Shape” which included the parameter of performance scores, performing times, and errors. We compared each parameter between the control group and the first session of training in the patient group (Comparison 1), between the control group and the last session of training in the patient group (Comparison 2), and between the first session of training in the patient group and the last session of training in the patient group (Comparison 3).

Table 3.

Program data obtained from control group, first session of the patients group, and last session of the patients group.

In the “Click” task, performance score showed a significant difference only in Comparison 3 (P < .05). Performing time showed significant differences in Comparison 1, 2, and 3 (P < .05). In the ‘Frying pan’ task, performance score showed significant differences in Comparison 1 and 3 (P < .05). Errors showed significant differences in Comparison 1 and 3 (P < .05). In the “Color” task, performance score showed significant differences in Comparison 1, 2, and 3 (P < .05). However, performing time showed no significant difference in all Comparisons (P > .05). In the “Bubble” task, performance score showed significant differences in Comparison 1 and 3 (P < .05). However, performing time showed no significant difference in all Comparisons (P > .05). In the “Shape” task, performance score showed a significant difference only in Comparison 1 (P < .05). However, performing time showed no significant difference in all Comparisons (P > .05). Errors showed significant differences in Comparison 1 and 3 (P < .05). In the “Figure” task, performance score showed significant differences in Comparison 1 and 3 (P < .05). However, performing time showed no significant difference in all Comparisons (P > .05). In the “Assembly” task, performance score showed significant differences in Comparison 1 and 3 (P < .05). Performing time showed a significant difference only in Comparison 1 (P < .05). In the “Memory” task, performance score showed significant differences in Comparison 1, 2, and 3 (P < .05). Performing time showed significant differences in Comparison 1 and 3 (P < .05). In the “Track” task, both performance scores and performing times showed significant differences in Comparison 1 and 2 (P < .05).

Additionally, we evaluated the task and parameters based on their ability to discriminate healthy individual and patients and to be responsive to training. First, we identified 3 task parameters considered ideal for not only training and also had evaluative value, as shown by significant differences in all of the Comparisons (Comparison 1, 2, and 3), such as Performing times of the “Click” task, Performance Scores of the “Color” task, and Performance Scores of the “Memory” task. There were 7 task parameters considered appropriate for training but had a low evaluative value, significant differences in Comparison 1 and 3, not in Comparison 2, such as Performance scores of the “Frying pan,” “Bubble,” “Figure,” and “Assembly” tasks, Errors of the “Frying pan” and “Shape” tasks, and performing time of the “Memory” task. There were 7 parameters whose task items themselves or level of difficulty needed to be re-adjusted because they failed to show differences in Comparison 1 or Comparison 3, such as Performance Scores of the “Click” and “Shape” tasks, and Performing times of the “Color,” “Bubble,” “Shape,” “Figure,” and “Assembly” tasks. Lastly, the “Track” task was categorized separately because it showed significant differences in Comparison 1 and 2, in both Performance Scores and Performing time, but it failed to show significance in Comparison 3. This result suggests that this task might have evaluative value, but it was not responsive to training.

4. Discussion

After a stroke, unilateral visual neglect or visuospatial, visuoconstructive disorder can occur, interferes with ADL, and is known to be associated with poor prognosis after stroke. The variable prevalence rates of unilateral visual neglect reported might be largely due to different assessment methods and timing of the evaluation. In this study, we used MVPT, line bisection test, and star cancellation test to diagnose unilateral neglect, however, there is no single “gold standard.” Computerized tests may be a novel alternative. Our goal here was to revise our previous program not only to improve the assessment of unilateral neglect but also for assessing the changes after rehabilitation within the program itself.

In this study, 9 individual tasks were intended to address 3 aspects of visual function: visual recognition, visual perception, and visual memory. “Click” and “Frying pan” tasks are for visual recognition, “Color,” “Shape,” “Figure,” and “Bubble” are for basic visual perception, “Track” and “Assembly” are for complex visual perception, and “Memory” is for visual memory. We also compared functional tests before and after training in the patient group, and MBI, MMSE, MVPT, Line bisection, Star cancellation, LOTCA, Digit F, and Digit B test showed significant improvement after training. These results were from the patients with visual neglect lasting >6 months post right hemispheric stroke, which differed from our previous study. In our previous study, we could not differentiate the effect of the program on visual neglect from spontaneous improvement because the subjects were in the relatively acute stage. Significant improvement not only in the tests for neglect such as MVPT, Line bisection, and Star cancellation test was demonstrated, but also the MBI test showed that there may also be motor benefits from encouraging patients to use the upper extremities more to perform the tasks even in the chronic stage of stroke. Our program had the additional effect of improving cognitive function, corroborating the positive result on the “Memory” task. During the training sessions, subjects were asked to remember and reproduce the trajectory of the targets. Through this task, subjects had the chance to improve their memory function. Our program is a computerized cognitive rehabilitation program and it was designed to give direct visual feedback of their own hands displayed on the screen to the subjects by using interactive motion tracking technology. So, patients can use either hand even if function is compromised. A joystick, mouse, or keyboard is not required to perform the tasks. Therefore, our program is simple to build compared with other programs using specific devices, including goggles or a specific station, and requires only a computer and camera. This simple structure also makes it easier for patients with disabilities. A simple structure is also important for us as an at-home rehabilitation program. Recently, there is a growing interest in home-based treatment for patients who cannot visit outpatient clinics frequently. Significant improvement of upper extremity function using an at-home rehabilitation program has been previously reported.^[5] Simple structure and low cost are desirable for at-home rehabilitation programs. Our program meets these specifications and could be used in at-home rehabilitation programs. Our program also has benefits of providing repetitive and controlled training opportunities for the patients with neglect after stroke. Moreover, our program is also thought to induce functional improvement by providing realistic feedback to the subjects’ hands during repetitive visual tasks. It offers rapid, real-time feedback from a mirror image of the patients’ own body, not from a virtual environment. The patient does not need to be immersed in the virtual environment, and is only required to concentrate on their upper extremities.

We developed a built-in scoring system to more objectively determine the effect of the program. These parameters were used to detect the changes in the patient group after training, and they also provided us with information on the characteristics of the tasks developed. There were 3 comparisons (Comparison 1, 2, and 3) among the groups (control group, first session of training in the patient group, and the last session of training in the patient group). Comparison 1 was between the control group and the first session of training in the patient group. This comparison showed differences between these groups, and therefore, it had evaluative value. Comparison 2 was between the control group and the last session of training in the patient group. If there were significant differences in this comparison, we assumed that the patient group still did not reach the level of the normal control group after the training sessions, which had evaluative value. Comparison 3 was between the first and the last sessions of training in the patient group, and hence, this comparison was for determining the effect of the training session using our program.

Based on these results, we categorized the parameters according to the statistical significance of the comparisons (Comparison 1, 2, and 3). Parameters in the first category (significant differences in Comparison 1, 2, and 3) had both evaluative value and were useful for determining the effect of training. Parameters in the second category (significant differences in Comparison 1 and 3, not in Comparison 2) were proper for determining the effect of training, but had lower evaluative value than the first category. These parameters might need to be adjusted to have a higher difficulty level in order to have evaluative value. For the parameters in the third category which failed to show differences in Comparison 1 or Comparison 3, we may need to adjust the items or the level of difficulty, assuming that failure to show the differences in Comparison 1 means that the level of difficulty is too low, and failure to show differences in Comparison 3 means that it is not responsible enough to show the effect of training. We categorized the “Track” task separately because it showed significant differences in Comparison 1 and 2, but not in Comparison 3. We suspect that the level of difficulty was too high in this task to produce training effect, although this task was proper for showing the differences between the control group and the patient group. To improve this task, the difficulty of the task should be decreased and/or an individualized level of difficulty could be provided to each patient.

To explain the natural recovery and the pathophysiology of neglect after stroke, recent studies have been used various imaging techniques such as diffusion tensor imaging (DTI),^[6] functional magnetic resonance image (fMRI),^[7] and magnetoencephalography.^[8] In these studies, the authors demonstrated that neglect syndrome is related to disruption in the connectivity of the attention network, and recovery from neglect syndrome is affected by restoration of this network. Many studies have been tried to explain the treatment mechanism of each therapeutic strategies for neglect also. Prism adaptation, which shifts the visual field laterally, has been widely used for treating lower sensorimotor plasticity, and it is also useful for treating neglect syndrome after stroke, because this sensorimotor plasticity is known to be linked with high-level cognitive function.^[9] To understand the mechanism of prism adaptation, a few studies have started to use the neurophysiologic methods such as event-related potentials (ERPs) based on electroencephalography (EEG) and a functional imaging study such as fMRI.^[10,11] However, these studies are still on the way to develop the methodology to understand the mechanism. Mechanism of mirror therapy for neglect is not clear yet.^[12] A few studies using fMRI suggest that mirror neuron system is activated by watching hand movement,^[13] and mirror therapy has been tried to treat neglect and motor function in stroke patients. Our program gives the patients repetitive visual stimulation to be tracked on the screen and offer visual feedback in mirror images for their hand movement, so we assumed that the mechanism of our program would be similar with those of prism adaptation and/or mirror therapy. However, further studies are needed to use advanced imaging studies or neurophysiologic studied in explaining the mechanism of these therapeutic strategies. Transcranial magnetic stimulation is also tried these days to improve unilateral neglect after stroke, and it is believed to enhance connections between the structures related to visual function.^[14] Computerized rehabilitation programs have been updated continually to keep up with advances in computer technology, and VR could be the most upgraded version. VR offers repetitive training environment that could address cognitive functions such as concentration and memory, and has been widely used to improve cognitive function after stroke.^[15] Many studies with individual programs using VR have reported benefits in unilateral neglect after stroke.^[16] Many considered the mechanism of VR for correction of visual neglect to be similar to that of prism adaptation because tasks in VR are able to change the patients’ representation of space.^[3] However, there are few if any studies to understand the mechanism by which VR can improve visual neglect. Until now, there is no consensus regarding the best neurophysiological or functional imaging technique identifying the mechanisms improving unilateral neglect after certain therapeutic strategies, and more studies on this topic are needed in future.

This study has several limitations. First, the sample size was relatively small. We recruited 16 patients and 19 healthy volunteers. Further study on additional patients is required. Second, we could not perform a double-blinded study. This study is a clinical trial with a program developed to improve unilateral neglect, and the results were compared between before and after the training session in the same patient group. In a further study, comparison between the treatment group and non-treatment group in the single-blinded setting is required.

Acknowledgment

The authors would like to thank, Dr. Diane Damiano, the Chief of the Functional and Applied Biomechanics Section in the Rehabilitation Medicine Department at the National Institutes of Health Clinical Center Intramural Research Program for her help in editing this manuscript.

Author contributions

Conceptualization: Don-Kyu Kim.

Data curation: Si Hyun Kang, Kyung Mook Seo.

Formal analysis: Don-Kyu Kim.

Investigation: Si Hyun Kang, Kwang Nam Choi.

Methodology: Kyung Mook Seo, Kwang Nam Choi.

Software: Kwang Nam Choi.

Validation: Kwang Nam Choi.

Writing – original draft: Si Hyun Kang, Don-Kyu Kim.

Writing – review & editing: Si Hyun Kang, Don-Kyu Kim, Kyung Mook Seo.