Validity and Reliability of Nintendo Wii Fit Balance Scores

Abstract

Context:

Interactive gaming systems have the potential to help rehabilitate patients with musculoskeletal conditions. The Nintendo Wii Balance Board, which is part of the Wii Fit game, could be an effective tool to monitor progress during rehabilitation because the board and game can provide objective measures of balance. However, the validity and reliability of Wii Fit balance scores remain unknown.

Objective:

To determine the concurrent validity of balance scores produced by the Wii Fit game and the intrasession and intersession reliability of Wii Fit balance scores.

Design:

Descriptive laboratory study.

Setting:

Sports medicine research laboratory.

Patients or Other Participants:

Forty-five recreationally active participants (age  =  27.0 ± 9.8 years, height  =  170.9 ± 9.2 cm, mass  =  72.4 ± 11.8 kg) with a heterogeneous history of lower extremity injury.

Intervention(s):

Participants completed a single-limb–stance task on a force plate and the Star Excursion Balance Test (SEBT) during the first test session. Twelve Wii Fit balance activities were completed during 2 test sessions separated by 1 week.

Main Outcome Measure(s):

Postural sway in the anteroposterior (AP) and mediolateral (ML) directions and the AP, ML, and resultant center-of-pressure (COP) excursions were calculated from the single-limb stance. The normalized reach distance was recorded for the anterior, posteromedial, and posterolateral directions of the SEBT. Wii Fit balance scores that the game software generated also were recorded.

Results:

All 96 of the calculated correlation coefficients among Wii Fit activity outcomes and established balance outcomes were interpreted as poor (r < 0.50). Intrasession reliability for Wii Fit balance activity scores ranged from good (intraclass correlation coefficient [ICC]  =  0.80) to poor (ICC  =  0.39), with 8 activities having poor intrasession reliability. Similarly, 11 of the 12 Wii Fit balance activity scores demonstrated poor intersession reliability, with scores ranging from fair (ICC  =  0.74) to poor (ICC  =  0.29).

Conclusions:

Wii Fit balance activity scores had poor concurrent validity relative to COP outcomes and SEBT reach distances. In addition, the included Wii Fit balance activity scores generally had poor intrasession and intersession reliability.

Key Points

• The Wii Fit balance activity scores had poor concurrent validity when compared with center-of-pressure outcomes from a 30-second single-limb stance and the anterior, posteromedial, and posterolateral reach distances from the Star Excursion Balance Test.

• Balance scores from the 12 tested Wii Fit activities generally had poor intrasession and intersession reliability and high minimal detectable change scores relative to the mean score of the activity.

• Wii Fit balance scores should not be used as an objective measure of progress during rehabilitation.

Athletic trainers and other allied health care professions have begun to use interactive gaming systems and technology to help rehabilitate patients with musculoskeletal and neurologic conditions.^1–4 Interactive games, such as DanceDance Revolution (Konami Corp, Toyko, Japan), and gaming systems, such as PlayStation 2 (Sony Computer Entertainment America, LLC, Foster City, CA) and Wii (Nintendo of America Inc, Redmond, WA), can be seen in physical therapy and sports medicine facilities. The growing popularity of interactive gaming partially is due to the belief that playing interactive video games during a rehabilitation session can direct a patient's focus away from the repetitive and mundane nature of rehabilitation exercises and toward the fun and competitive aspects of the video game.^5–7 Thus, interactive gaming technology subjectively improves adherence rates for rehabilitation programs⁶ and presents allied health care providers with an adjunct to traditional rehabilitation exercises.

However, Nintendo recently released the Wii Balance Board (Nintendo of America Inc) as a controller for games such as Wii Fit (Nintendo of America Inc) and Wii Fit Plus (Nintendo of America Inc). The Wii Balance Board possesses characteristics similar to a research-grade force plate and contains transducers used to assess force distribution and the subsequent movements of an individual's center of pressure (COP). Thus, the Wii Balance Board combines current force-platform technology with the entertainment value of gaming while completing fitness activities, such as balance and yoga. Furthermore, the Wii Fit and Wii Fit Plus games include personalized settings to track an individual's progress over time. The completion of these activities allegedly improves balance, and some health care providers have hypothesized that components of interactive games, such as balance scores and player ratings, might be related to sensorimotor function.⁶

Thus, using the Wii Balance Board while playing Wii Fit or Wii Fit Plus could be an effective assessment tool for athletic trainers and other allied health care providers because the board and games have the potential to improve balance and provide objective measures of balance over time to monitor rehabilitation progress. Clark et al⁸ compared COP data from a research-grade force plate with COP data from a Wii Balance Board. To do this, the authors interfaced the Wii Balance Board with a laptop computer using custom-written software, bypassing game software and scores. The results indicated that a Wii Balance Board produced raw COP means that were similar to the means that the force plate produced, and it had good to excellent test-retest reliability.⁸ However, the creation and use of custom-written software similar to that used by Clark et al⁸ is not practical in the clinical setting. Furthermore, doubt exists about the validity and reliability of the balance scores that arise from the software implemented in the Wii Fit and Wii Fit Plus games. Until this information is elucidated, clinicians using balance scores from the Wii Fit or Wii Fit Plus games to monitor progress during rehabilitation might be making clinical decisions based on inaccurate or unreliable data. Therefore, the purpose of this investigation was to determine the concurrent validity and the intrasession and intersession reliability of balance scores the Wii Fit game produces.

METHODS

Design and Participants

For this descriptive laboratory study, I used a test-retest study design to achieve the stated aims. A total of 45 participants (age  =  27.0 ± 9.8 years, height  =  170.9 ± 9.2 cm, mass  =  72.4 ± 11.8 kg) completed 2 test sessions that were separated by 1 week. All participants were between 18 and 57 years of age and were recreationally active, which operationally was defined as participating in aerobic activities at least 3 times each week for at least a total of 90 minutes. In addition, participants had no known visual, vestibular, and neurologic impairments; had not had a concussion for at least 3 months before testing; and were not taking medication that might impair balance for at least 1 month before testing. However, no limitations were placed on the presence of acute (eg, lateral ankle sprain, anterior cruciate ligament [ACL] rupture) or chronic (eg, chronic ankle instability, ACL reconstruction) lower extremity conditions before enrollment in the study so that a more heterogeneous sample would be captured. A heterogeneous sample was used to ensure that reliability and validity would be established regardless of the type of participant tested.^9,10 To quantify disability associated with acute and chronic lower extremity conditions, all participants completed 2 questionnaires: a basic lower extremity medical history questionnaire and the Lower Extremity Functional Scale (LEFS).¹¹ In total, 22 people had histories of at least 1 lower extremity musculoskeletal condition, with 6 participants reporting multiple conditions (ankle sprain  =  6, ankle instability  =  4, ACL rupture  =  5, meniscal injury  =  2, lower extremity fracture  =  5, other conditions  =  6 [eg, low back pain, tendinitis]). People without histories of lower extremity injuries had an average LEFS score of 78.4 ± 2.3, whereas people with histories of lower extremity injuries had an average LEFS score of 72.4 ± 8.7 out of 80 possible points. In research on the LEFS questionnaire, investigators^11,12 have shown good clinometric properties across a wide range of musculoskeletal disorders of the lower extremity. All participants provided written informed consent before completing the test sessions, and the study was approved by the Institutional Review Board of the University of North Carolina at Charlotte.

Procedures

All participants reported to the Biodynamics Research Laboratory for both test sessions. Upon reporting for the first test session, they were exposed to and allowed to practice each of the balance tasks associated with the research protocol. This practice time was followed by a mandatory 10-minute rest period during which participants completed the medical history and LEFS questionnaire. They completed 2 test trials per limb, if applicable, of all balance tasks with 30-second rest periods between trials and 1-minute rest periods between tasks. Established balance tasks, which were completed during the first test session only, included a single-limb– stance task on a force plate and the Star Excursion Balance Test (SEBT).¹³ The established measures of static and dynamic balance used in our investigation were chosen for the following reasons: (1) they have been shown to be reliable,^13,14 (2) they have been able to discriminate between groups with known balance impairments and controls,^15–17 (3) they commonly have been reported in the literature,^13–17 and (4) they were deemed to be functionally appropriate for the sample tested. Wii Fit balance activities were completed during the first and second test sessions. The order of test completion during each test session was randomized for each participant before the start of data collection to minimize potential effects of learning and fatigue. The second test session was completed 1 week after the first test session.

Established Balance Tasks

For the single-limb–stance task, participants stood barefoot on 1 limb on an instrumented force plate with hands on hips and eyes open.^18,19 Participants completed two 30-second trials per limb on a triaxial force plate (model 4060NC; Bertec Corporation, Columbus, OH) mounted flush with the surrounding floor. Ground reaction force data were collected at 200 Hz and filtered using a fourth-order, zero-lag, low-pass filter with a cutoff frequency of 5 Hz. The chosen outcome measures were average sway in the anteroposterior (AP) and mediolateral (ML) directions and the AP, ML, and resultant COP excursions. Given that the single-limb–stance task trials were the same duration, the AP, ML, and resultant COP excursions are analogous to the average AP, ML, and resultant COP velocities (excursion divided by time). A 30-second time frame was chosen to keep the duration of the task similar to the duration of the Wii Fit balance activities.

Participants also completed 2 reach trials per limb in 3 SEBT reach directions (anterior, posteromedial, and posterolateral). Testing only these 3 directions is recommended because of the redundancy in information obtained when all 8 directions are tested.¹⁶ To conduct the reach trials, each participant placed the stance limb in the center of the SEBT grid and placed his or her hands on the hips. Next, he or she reached as far as possible in the 3 reach directions with the contralateral limb. Reach distance was defined as the farthest point that an individual could touch without accepting weight and while maintaining balance through the return to a bilateral stance.^13,16,20,21 All reach distances were normalized to the participant's leg length (distance from the anterosuperior iliac spine to the ipsilateral medial malleolus). The SEBT is a reliable measure of dynamic balance (intraclass correlation coefficient [ICC] [2,1]  =  0.85–0.96).¹³

Wii Fit Balance Activities

Twelve Wii Fit activities were chosen to validate as many potential assessment tools as possible (Table 1). These activities consisted of a variety of dual-limb and single-limb support tasks that were both static and dynamic. All tasks were conducted using a standard Wii game console, Wii Balance Board, and Wii Fit software and were completed in accordance with the game software. The game console was connected to a projector (model XD490U DLP; Mitsubishi Electric Visual Solutions America, Inc, Irvine, CA) so that the gaming screen was 2 m in front of the participants and the center of the screen was at eye level. During each test session, 7 Wii Fit activities were completed (deep breathing, tree, standing knee, palm tree, single-leg extension, single-leg twist, sideways leg lift) twice per limb. The remaining activities (basic balance, agility, walking, steadiness, and single-leg stance) are part of the Wii Fit's body test, and the game software randomly selects 2 of the 5 activities to be completed per body test. To collect as much data as possible, 2 body tests were completed for each test session. Thus, each participant completed 5 body test activities (basic balance, agility, walking, steadiness, and single-leg stance) per test session. The Figure illustrates the testing sequence for the Wii Fit activities. Two trial averages were not always available for the body test activities because of the random selection made by the game software. In such cases, the single trial score was used for further analysis.

Table 1.

Wii Fit^a Activities

Figure.

Flow of the data collection sequence for each test session. ^a Indicates completion of the single-leg stance and Wii Fit (Nintendo of America Inc, Redmond, WA) test and Star Excursion Balance Test were randomized for each participant during test session 1. ^b Indicates that during each test session, the Wii Fit activities, which included 2 body tests and 8 other activities, also were randomized for each participant.

Statistical Analysis

Wii Fit balance activity scores initially were submitted to separate independent-samples t tests. These analyses determined that no differences existed between the left and right limbs of individuals who reported no history of lower extremity conditions when applicable (P > .05) and no differences existed between the uninjured limbs and the limbs of the participants who self-reported a history of lower extremity injury (P > .05). Therefore, data from both the left and right limbs of all participants were used to calculate concurrent validity and intrasession and intersession reliability.

To determine concurrent validity, Pearson product moment correlation calculations were used to assess the strength of the relationships between the Wii Fit balance activity scores and the COP and SEBT outcome measures. The quality of the correlation was interpreted as follows: less than 0.50 indicated poor validity; 0.50 to 0.75, moderate to good validity; and more than 0.75, excellent validity.²² Intrasession and intersession (1-week) reliability was quantified using dependent-samples t tests and ICCs (2,1). The precision of each Wii Fit activity outcome measure was calculated using the standard error of measurement (SEM). In addition, minimal detectable change (MDC) scores were used to quantify how much a Wii Fit balance score would need to change after a rehabilitation intervention for that change to be considered confidently a true change (ie, improvement or degradation) due to the intervention and not due to measurement error.²³ Reliability coefficients were interpreted as follows: less than 0.69 was considered poor; 0.70 to 0.79, fair; 0.80 to 0.89, good; and 0.90 to 1.00, excellent.²² The α level was set a priori at ≤.05 to determine if (1) the relationships were correlated, (2) the within-sessions trials were different, and (3) the between-sessions trial averages were different.

RESULTS

The group means, standard deviations, and number of limbs that completed a task during the first test session are presented in Table 2. The Pearson product moment correlations indicated that 4 relationships were correlated (Table 3). However, all 96 of the calculated relationships between Wii Fit balance activity scores and established balance task outcomes, even those identified as correlated, were interpreted as poor (<0.50). Thus, concurrent validity between Wii Fit balance scores and established balance tasks outcome was not established. Indeed, the highest correlation coefficient was between the Wii Fit walking activity score and the ML (r  =  0.32, P  =  .045) and resultant (r  =  0.32, P  =  .05) COP excursion outcomes.

Table 2.

Performance on Established and Wii Fit Balance Tasks (Mean ± SD)

Table 3.

Relationships Between Wii Fit^a Activities and Established Balance Tasks

Intrasession and intersession reliability estimates, including t and P values, precision (SEM), and MDC scores, for the Wii Fit balance scores are presented in Tables 4 and 5, respectively. None of the Wii Fit balance activity scores differed between trials 1 and 2 during the first test session (Table 4). One score from the Wii Fit balance activity (single-leg stance) received an intrasession ICC point estimate that was interpreted as good. Three balance scores (steadiness, deep breathing, and single-leg twist) had intrasession ICC point estimates that were interpreted as fair, whereas the remaining 8 activities had intrasession ICC point estimates of poor reliability. Similarly, 11 of the 12 Wii Fit balance scores demonstrated poor intersession reliability, whereas deep breathing demonstrated a fair reliability point estimate. Four other balance scores (agility, single-leg extension, single-leg twist, sideways leg lift) differed between the first and second test sessions, with scores improving during the second test session in all 4 activities. High MDC scores, which represent in units of the outcome measure the minimal change that must occur in the Wii Fit activity before the clinician can be truly confident that the change is not due to error, also indicated poor stability of Wii Fit balance scores.

Table 4.

Intrasession Reliability Estimates for Wii Fit Balance^a Activity Scores

Table 5.

Intersession Reliability Estimates for Wii Fit Balance^a Activity Scores

DISCUSSION

The purpose of this investigation was to determine the suitability of Wii Fit balance activity scores as objective assessment tools to monitor rehabilitation progression. Indeed, an inexpensive (relative to traditional force platforms), portable, valid, and reliable way to objectively assess balance would be valuable to a wide range of allied health care providers.⁸ The most important findings of this investigation were that Wii Fit balance scores possessed poor levels of concurrent validity and generally poor intrasession and intersession reliability estimates, as well as high MDC scores. These findings are contrary to those of Clark et al,⁸ who demonstrated that the raw COP data collected by a Wii Balance Board had reliability estimates comparable with those collected by a traditional research laboratory force platform. However, they bypassed the Wii software by interfacing the Wii Balance Board with a laptop computer using custom-written software, which limits the direct comparisons between this investigation and their work.⁸

Poor levels of concurrent validity (ie, Wii Fit balance scores were correlated poorly with balance scores from established balance tasks) mean that clinicians cannot predict accurately how well an individual will perform on a Wii Fit activity based on COP outcomes during a 30-second single-leg stance or his or her reach distance during an SEBT or vice versa. The established measures of balance that I used (single-leg stance on a force plate and SEBT) are not considered the criterion standard for static and dynamic balance. However, a criterion standard for static or dynamic balance has not been established in the literature. In addition, several of the Wii Fit activities are performed during dual-limb support, but both of the established measures used are completed during single-limb support. Thus, the current established measures of balance might not have been the most appropriate tasks from which to establish concurrent validity for some Wii Fit activities, which was a limitation of the investigation. However, a static dual-limb support task was not chosen because static dual-limb support tasks are not considered challenging enough to evaluate postural control or monitor the rehabilitation progress of physically active individuals with lower extremity musculoskeletal conditions.

Another potential reason for the poor validity estimates observed in this investigation is the algorithm used by the Wii Fit software to calculate the balance scores. Currently, the exact algorithms used to calculate Wii Fit balance scores are unknown to the general public, but regardless of the calculation details, the software would apply the algorithm the same way for each trial, thereby giving standard outcomes for each balance board activity. Thus, the Wii Fit software might explain the poor concurrent validity, but the gaming software is not likely responsible for the poor reliability estimates observed. In the future, researchers should compare the Wii Fit dual-limb support tasks with an established dual-limb support task and attempt to determine concurrent validity with other established static and dynamic balance tasks (eg, Berg Balance Scale, Balance Error Scoring System, hop to stabilization) and outcomes (eg, time to boundary, time to stabilization). In addition, concurrent validity should be confirmed in various age groups and clinical populations through mutually exclusive investigations (ie, separately determine concurrent validity in people with acute ankle sprains, chronic low back pain, and other lower extremity conditions). Using this systematic approach in the future will provide for each injured group more statistical power than and expand on the clinical applicability of the results obtained from the current heterogeneous sample.

The general poor reliability estimates and high MDC scores strongly suggest that clinicians should not use the balance scores produced by Wii Fit activities as an objective tool to monitor progress during rehabilitation. Indeed, the generally high MDC scores indicate that a substantial and impractical change in Wii Fit balance scores is needed to exceed the error of the measurement. For example, a change equal to roughly 50% of the recorded mean for the tree activity is needed to be confident that intersession change (Table 5) is due to the intervention delivered and not to the measurement error.

A plausible cause of the generally poor intrasession and intersession reliability involves the novelty of the Wii Fit activities to the participants. In this investigation, participants were allowed to practice each Wii Fit activity once (per test session) before completing the test trials. One practice trial per activity creates potentially insufficient self-confidence and does not allow participants to feel comfortable with the tasks, and it should be considered a limitation of the investigation. The balance score improvements between the first and second test sessions in the agility, single-leg extension, single-leg twist, and sideways leg lift tasks provide evidence that supports this hypothesis. Increasing the number of trials used to calculate the Wii Fit activity averages possibly could influence positively both intrasession and intersession reliability estimates. For example, a standard protocol with multiple practice trials and at least 3 test trials used to calculate average scores might improve the reliability of the Wii Fit activities, as previously reported with the SEBT.¹³ Thus, the reliability and subsequently the concurrent validity of Wii Fit balance scores might improve as a user's experience with the game activities increases or as a greater number of trials serve as the basis for the averages used for analysis. However, future investigations are needed to confirm or refute this hypothesis. Research also is needed to confirm both the intrasession and intersession reliability of the Wii Fit activities associated with the body test activities (ie, basic balance, agility, walking, steadiness, and single-leg stance). Indeed, during this investigation, only a few participants (as few as 5) completed these activities, which limits the power available to quantify reliability estimates and the ability to draw solid conclusions from the data.

I undertook this investigation because of the growing use of interactive gaming technology, in particular the Wii and Wii Fit, by allied health care providers and especially athletic trainers. The results suggest that Wii Fit balance scores, as produced by the game software, should not be used as tools to assess or monitor progress during rehabilitation. If future researchers confirm these findings, several emerging techniques might enable allied health care providers to take advantage of the portability and cost advantages of a Wii and Wii Balance Board while overcoming the apparent poor reliability and validity observed in this investigation.^8,24–27 In general, these techniques involve bypassing the scores produced by the game software and simply using the raw data from the Wii Balance Board or other peripheral devices. However, without specific skill sets, use of these techniques likely will require collaboration with units or individuals familiar with computer programming. Most importantly, the findings do not condemn the use of a Wii, Wii Balance Board, or other interactive gaming system by allied health care providers. Indeed, the use of interactive gaming technologies as tools or exercise in the rehabilitation process is recommended because of the large number of purported benefits (eg, improving balance,⁶ improving compliance⁶) associated with using interactive gaming technology.

CONCLUSIONS

Wii Fit balance scores had poor concurrent validity compared with the COP outcomes from a 30-second single-limb stance and the anterior, posteromedial, and posterolateral reach distances from the SEBT. In addition, balance scores associated with the 12 Wii Fit activities tested in this investigation generally had poor intrasession and intersession reliability and generally high MDC scores relative to the mean score of the activity. However, future research is needed to confirm the reliability of those activities associated with the Wii Fit body test. These findings indicate the Wii Fit balance scores should not be used as an objective measure of rehabilitation progress, but using a Wii or Wii Fit in a clinical setting is still recommended based on the other purported benefits.