Abstract
Purpose
To evaluate the test-retest reliability, concurrent validity and responsiveness of the Wheelchair Skills Test - Questionnaire (WST-Q) Version 4.1 for powered wheelchair users.
Methods
A volunteer sample of 72 community-dwelling, experienced powered wheelchair users, ranging in age from 50 to 77 years, participated in this study. Participants completed measures at baseline and 1 month later.
Results
Mean ± standard deviation total percentage WST-Q scores at baseline and 1 month were 83.7% ±10.9 and 86.3% ±10.0 respectively. Cronbach’s alpha was 0.90 and the 1 month test-retest intraclass correlation coefficient (ICC1,1) was 0.78 (confidence interval:0.68–0.86). There were no floor or ceiling effects. Percentages of agreement between baseline and 1 month for individual skills ranged from 72.2–100%. The correlations between the WST-Q and the objective Wheelchair Skills Test (WST), WheelCon and Life Space Assessment were r=0.65, r = 0.47 and r = 0.47 respectively. The standard error of measurement (SEM) and smallest real difference (SRD) were 5.0 and 6.2 respectively.
Conclusion
The WST-Q 4.1 has high internal consistency, strong test-retest reliability and strong support for concurrent validity and responsiveness.
Keywords: Wheelchairs, Outcome assessment (health care), Rehabilitation
INTRODUCTION
In 2010, based on the US Census Bureau’s Survey of Income and Program Participation, there were approximately 3.6 million community-dwelling wheelchair users in the United States.1 Recent estimates indicate that between 17–30%2 of all wheelchair users use a power wheelchair or scooter. Provision of a power wheelchair is a common rehabilitation intervention for individuals with significant mobility disability and has been shown to increase independence with mobility and ease of mobility-related participation.3 However, power wheelchair use can be limited by many barriers including uneven footpaths, uneven ground, curbs, narrow doorways and aisles, potholes, stairs, hills and crowded places.4 Consequences related to these barriers include the need for assistance from family members or caregivers, concerns about safety, decreased participation and injury.4–9
Prescribing a power wheelchair that minimizes the impact of environmental barriers and matches the individual’s capabilities, needs, preferences and chosen activities can be a complicated process. The complexity of this process is heightened by the inherent safety concerns associated with power wheelchairs.10 Assessment is a key step in the World Health Organization’s wheelchair service-delivery process.11 The need for validated assessment tools is apparent.12
However, clinicians often rely on assessments of an individual’s attributes (e.g., cognition, vision) that may lack predictive validity with respect to an individual’s suitability for power wheelchair use.10 This type of assessment may do an injustice to individuals who initially may not appear to be capable of learning to use a power wheelchair. A standardized assessment of wheelchair skills can identify individuals who, with training, may be suitable power wheelchair candidates.13,14 Use of standardized assessments can help to evaluate baseline skill level, identify wheelchair skill goals and assess change in wheelchair skill over time.
The Wheelchair Skills Program (WSP) includes wheelchair skills assessments for wheelchair users and caregivers of manual wheelchairs, power wheelchairs and scooters.15 The Wheelchair Skills Test (WST) is an objective measure and the Wheelchair Skills Test Questionnaire (WST-Q) is a subjective, self-report measure.16 Initially developed for manual wheelchair use, the measurement properties of the WST for manual wheelchair users have been well documented17–20 and the WST has been shown to correlate highly with the WST-Q.21–23 While there are merits and limitations to both the objective and subjective versions,24 the WST-Q may be the most appropriate assessment of wheelchair skills when objective testing is impractical or impossible. The measurement properties of the WST-Q for powered wheelchair users have not yet been evaluated.
The purpose of this study was to evaluate some of the measurement properties of the WST-Q for powered wheelchair users. We hypothesized that the WST-Q would: (i) have high internal consistency; (ii) have high test-retest reliability; (iii) have positive moderate correlations with relevant existing measures (concurrent validity); and (iv) accurately detect change when it has occurred (responsiveness).
METHODS
Study Design
This study used a one-month test-retest design. The data were collected as part of a 1-year longitudinal study investigating the natural history of power wheelchair use among older adults over time.
Participants
To assess our a priori hypothesis that the WST-Q would have moderate correlations with other relevant variables, given an α of 0.05 and a β of 0.80, we determined that a minimum of 68 participants would be needed to provide sufficient power.25 Our sample of convenience consisted of 72 power wheelchair users.
Recruitment and Screening
Participants were recruited from 6 Canadian cities: Vancouver, British Columbia; London, Ontario; Toronto, Ontario; Montreal, Quebec; Quebec City, Quebec; and Halifax, Nova Scotia. Recruitment strategies included sending information, via letter, to potential participants through rehabilitation research databases, clinicians, and seating and mobility services. Advertisements were also posted in locations frequented by power wheelchair users, including medical equipment dealers and community centers, as well as via e-bulletins through community organizations, such as the Canadian Paraplegic Association and peer support groups. To be included in the study, participants had to be at least 50 years of age (an inclusion criterion to be included in the larger longitudinal study that focused on older adults), have at least 6 months of power wheelchair experience, be able to operate their power wheelchairs independently, be using the same power wheelchair for the baseline and follow up assessment, and be able to speak and write in English or French.
Ethical Issues
We obtained ethical approval for each data collection site through the local university or hospital research ethics boards. Each participant provided informed consent.
Wheelchair Skills Test Questionnaire (WST-Q)
We used the WST-Q 4.1 for powered wheelchair users in this study.16 This self-report test evaluates 32 skills (listed later) ranging from turning the power wheelchair on/off to ascending/descending a 5cm level change. It was administered and scored according to the WST Manual.16 Although the WST-Q can be used to measure both capacity (what the individual can do) and performance (what the individual does do),26 we assessed only capacity in this study. Specifically, the WST-Q was conducted using a semi-structured interview, whereby the individual was asked whether he/she believed himself/herself to be capable of performing each wheelchair skill (capacity). Each skill was scored using a dichotomous response format (pass/fail) according to explicit scoring criteria. Total WST-Q percentage scores were calculated (number of passed skills/number of possible skills × 100%).
Comparison Outcome Measures
All comparison outcome measures included in this study have previously been used with wheelchair users and are described briefly below.
Life Space Assessment (LSA)
The LSA measures frequency and independency of mobility across a continuum of environments through which a person reports moving during the prior 4 weeks, including within the home, around the home, in the neighborhood, in town and outside of town.27,28
Wheelchair Use Confidence Scale for powered wheelchair users (WheelCon-P)
The WheelCon-P is a 59-item self-report questionnaire that measures confidence with power wheelchair use.29,30 Sixty-three percent of the WheelCon-P items are related to negotiating the physical environment.
Wheelchair Skills Test (WST) for powered wheelchair users
The WST 4.1 is a 32-item objective test of wheelchair skills.16 It evaluates the same 32 skills as the WST-Q and is scored in the same way.
Procedure
Participants who met the study inclusion/exclusion criteria attended data collection appointments at baseline and 1 month later. At baseline, participants completed a demographic questionnaire and the 4 additional measures (described above) in the following order: the LSA, WheelCon-P, WST-Q and WST. There were 6 testers, one at each data collection site. Each tester received training in administering and scoring all outcome measures from the primary investigator (PWR).
Data Analysis
We used the Statistical Package for the Social Sciences 21.0 for the data analyses (IBM Corp. Released 2012. IBM SPSS Statistics for Mac, Version 21.0. Armonk, NY: IBM Corp.). Descriptive statistics and total percentages were calculated for the total WST-Q capacity scores. Normal distribution of the data was tested with the Kolmogorov-Smirnov test. The percentages of individuals with the lowest (0%) and highest (100%) scores were determined and values greater than 20% were considered as floor and ceiling effects.31 The analyses for the specific components of the study are described below. The level of statistical significance was set at p < 0.05.
Internal consistency was calculated using Cronbach’s α. One-month test-retest reliability of the WST-Q was evaluated using an intraclass correlation coefficient (ICC1,1)with 95% confidence intervals (CI), calculated using one-way analysis of variance (ANOVA).25 We also calculated the success rates and the percentage agreement for individual WST-Q skills. The percentage agreement for each skill was calculated by comparing each participant’s baseline and 1-month score and dividing by the number of participants who were tested on the skill (i.e., excluding the participants for which their power wheelchair did not have the part or function). We defined ≥75% agreement as clinically significant.22,23 The Bland-Altman limits of agreement plot was used to provide a visual assessment of how individual WST-Q scores varied between baseline and 1 month.32
To assess concurrent validity, associations between the total percentage WST-Q scores and the comparison outcome measures were calculated using Pearson’s Product Moment Correlation. We also compared individual wheelchair skills between the WST and WST-Q scores using percentage agreements, as described above.22,23 The strength of the associations was interpreted according to Domholdt’s classification.33 The standard error of measurement (SEM)25 and smallest real difference (SRD)34 were used to measure responsiveness.
RESULTS
Participants
The participants’ demographic, clinical and wheelchair-use characteristics are presented in table 1. This sample of older adults was equally distributed between men and women and there was a wide range of power wheelchair experience. Among the range of primary diagnoses present in this sample, spinal cord injury and multiple sclerosis were most common.
Table 1.
Participants’ Demographic, Clinical and Wheelchair-Use Characteristics
| Characteristic | Value |
|---|---|
|
| |
| Age in years, mean (SD) | 60.7 (7.3) |
| Range | 50–77 |
|
| |
| Men (n, %) | 36 (50.0%) |
|
| |
| Language (n, %) | |
| English | 53 (73.6%) |
| French | 18 (25.0%) |
| Other | 1 (1.4%) |
|
| |
| Marital Status (n, %) | |
| Married/common-law | 28 (38.8%) |
| Single | 25 (34.7%) |
| Separated/divorced/widowed | 19 (26.3%) |
|
| |
| Living Situation (n, %) | |
| Alone | 38 (52.8%) |
| With spouse | 24 (33.3%) |
| With others | 10 (13.9%) |
|
| |
| Education (n, %) | |
| Less than high school | 9 (12.6%) |
| High school | 12 (16.7%) |
| More than high school | 51 (70.9%) |
|
| |
| Employment Status (n, %) | |
| Employed | 4 (5.6%) |
| Unemployed | 18 (25.0%) |
| Retired | 46 (63.9%) |
| Volunteer | 21 (29.2%) |
|
| |
| Diagnosis (n, %) | |
| Multiple sclerosis | 17 (23.6%) |
| Spinal cord injury | 14 (19.4%) |
| Muscular Dystrophy | 5 (6.9%) |
| Stroke | 4 (5.6%) |
| Arthritis | 4 (5.6%) |
| Lower limb amputation | 2 (2.8%) |
| Other | 26 (36.1%) |
|
| |
| Years with diagnosis | |
| Median | 25.8 |
| Range | 1.8–72.6 |
|
| |
| Years using power wheelchair | |
| Median | 10.0 |
| Range | 0.5–39.0 |
|
| |
| Primary Mobility Device (n, %) | |
| Power wheelchair | 62 (86.1%) |
| Manual wheelchair | 7 (9.7%) |
| Other | 3 (4.2%) |
|
| |
| Powered wheelchair frame (n, %) | |
| Front wheel drive | 4 (5.6%) |
| Mid wheel drive | 40 (55.6%) |
| Rear wheel drive | 28 (38.9%) |
|
| |
| Power Wheelchair Use (n, %) | |
| At home | 54 (75.0%) |
| In the community | 71 (98.6%) |
| For leisure activities | 47 (65.3%) |
Note: For the employment status and power wheelchair use variables, participants had the option to choose all answers that apply. Also, for continuous data, we report the mean value (±SD) when the data were normal. When the data were non-normal, we report the median value and the range.
WST-Q Scores
The WST-Q mean administration time ± standard deviation (SD) was 16.3 ± 5.2 and 15.7 ± 9.5 minutes for baseline and 1 month respectively. The total percentage WST-Q capacity scores demonstrated a normal distribution, as confirmed by the Kolmogorov-Smirnov (p > 0.05). The mean ± SD for the WST-Q capacity total percentage scores at baseline and 1 month were 83.7% ±10.9 and 86.3% ±10.0. In terms of ceiling effects, 5 (6.9%) and 7 (9.7%) participants scored 100% at baseline and 1 month respectively. There were no floor effects (scores of 0) present.
Reliability
Cronbach’s alpha coefficient was 0.90. The one-month test-retest reliability ICC1,1 was 0.78 with a confidence interval of 0.68–0.86. A scatter plot of the total WST-Q capacity scores at baseline and 1 month is shown in figure 1. The Bland-Altman plot (not shown) displayed the 95% limits of agreement (9.9 to −15.1), the WST-Q mean difference between baseline and 1 month (− 2.6), and a fairly equal distribution of scores above and below the mean difference. The limits of agreement represent how far apart the WST-Q scores were likely to be for most individuals and the distribution of scores suggested no systematic bias in score between administrations. There were 5 outliers who did not fall between the 95% limits of agreement. One outlier had a score higher than the 9.9 limit of agreement and 4 outliers had scores less than the −15.1 limit of agreement.
Figure 1.
Total WST-Q percentage scores plotted at baseline and 1 month (ICC=0.78)
Table 2 presents the percentage success rate and agreement for individual skills between the WST-Q at baseline and 1 month. Success rates for individual skills ranged from 22.2% to 100%. The skills with success rates below 70% represented a mix of indoor, community and advanced skills and included: ‘gets from ground to wheelchair’, ‘disengages and engages motors’, ‘transfers from wheelchair to bench and back’, ‘operates battery charger’, ‘ascends 5cm level change’, ‘gets through hinged door’ and ‘picks object up from floor’. Percent agreement between the WST-Q scores at baseline and 1 month ranged from 72.2% to 100%. The skill for which the agreement was 72.2%, ‘gets over 15cm pothole’, was the only skill that did not meet the threshold of 75% or greater that we had defined for a clinically significant agreement.
Table 2.
WST-Q 4.1 Percentage Success Rates and Agreement for Individual Wheelchair Skills Between Baseline and 1 Month
| Skill | WST-Q Time 1 |
WST-Q Time 2 |
% Agreement | ||
|---|---|---|---|---|---|
| N | % Success | N | % Success | ||
| 1. Moves controller away and back | 59 | 98.3 | 60 | 98.3 | 94.4 |
| 2. Turns controller on and off | 72 | 95.8 | 72 | 97.2 | 98.6 |
| 3. Selects drive modes and speeds | 72 | 100 | 72 | 100 | 100 |
| 4. Controls tilt function | 48 | 97.9 | 47 | 97.8 | 98.6 |
| 5. Control recline function | 7 | 85.7 | 6 | 83.3 | 97.2 |
| 6. Disengages and engages motors | 71 | 36.6 | 72 | 38.8 | 86.1 |
| 7. Operates battery charger | 72 | 61.1 | 72 | 66.6 | 93.0 |
| 8. Rolls forward 10m | 72 | 100 | 72 | 100 | 100 |
| 9. Rolls forward 10m in 30 sec | 72 | 97.2 | 72 | 100 | 97.2 |
| 10. Rolls backward 5m | 72 | 94.4 | 72 | 100 | 94.4 |
| 11. Turns 90° while moving forward | 72 | 100 | 72 | 100 | 100 |
| 12. Turns 90° while moving backwards | 72 | 91.6 | 72 | 95.8 | 90.2 |
| 13. Turns 180° in place | 72 | 98.6 | 72 | 97.2 | 95.8 |
| 14. Maneuver sideways | 72 | 97.2 | 72 | 94.4 | 91.6 |
| 15. Gets through hinged door | 72 | 65.2 | 72 | 72.2 | 81.9 |
| 16. Reaches 1.5m high object | 72 | 81.9 | 72 | 93.0 | 88.8 |
| 17. Picks object from floor | 72 | 69.4 | 72 | 75.0 | 86.1 |
| 18. Relieves weight from buttocks | 72 | 86.1 | 72 | 87.5 | 83.3 |
| 19. Transfers to bench and back | 72 | 54.1 | 72 | 57.5 | 91.6 |
| 20. Rolls 100m | 72 | 100 | 72 | 100 | 100 |
| 21. Avoids moving obstacles | 72 | 94.4 | 72 | 98.6 | 95.8 |
| 22. Ascends 5° incline | 72 | 100 | 72 | 100 | 100 |
| 23. Descends 5° incline | 72 | 100 | 72 | 100 | 100 |
| 24. Ascends 10° incline | 72 | 91.6 | 72 | 97.2 | 88.8 |
| 25. Descends 10° incline | 72 | 93.0 | 72 | 98.6 | 94.4 |
| 26. Rolls 2m across 5° side slope | 72 | 88.8 | 72 | 97.2 | 88.8 |
| 27. Rolls 2m on soft surface | 72 | 76.3 | 72 | 80.5 | 77.7 |
| 28. Gets over 15cm pothole | 72 | 70.8 | 72 | 68.0 | 72.2 |
| 29. Gets over 2cm threshold | 72 | 91.6 | 72 | 91.6 | 91.6 |
| 30. Ascends 5cm level change | 72 | 62.5 | 72 | 66.6 | 79.1 |
| 31. Descends 5cm level change | 72 | 80.5 | 72 | 86.1 | 88.8 |
| 32. Gets from ground to wheelchair | 72 | 22.2 | 72 | 23.6 | 93.0 |
WST-Q = Wheelchair Skills Test Questionnaire
Note: Skills for which the sample size was less than n=72 were either the result of the participant’s power wheelchair not having the part or function (e.g., no tilt or recline function), participant error, or tester error.
Concurrent Validity
Correlations between the WST-Q and other variables are presented in Table 3. All correlations were in the direction hypothesized and statistically significant correlations were found between the total percentage WST-Q capacity scores and life space travelled (LSA), wheelchair confidence (WheelCon-P), and objective wheelchair skills (WST). The highest correlation found was between the WST-Q and the WST (r=0.65).
Table 3.
Mean Scores and Correlations for the Study Measures
| Study Measures (maximum score) | Mean (SD) | Correlation (r) with WST-Q, p value |
|---|---|---|
| WST-Q (baseline) ( /100) | 83.6 (10.9) | N/A |
| LSA (assisted life space) ( /5) | 2.0 (0–5) | 0.47, p<0.001 |
| WheelCon-P ( /100) | 76.6 (13.5) | 0.47, p<0.001 |
| WST ( /100) | 80.1 (10.5) | 0.65, p<0.001 |
WST-Q = Wheelchair Skills Test Questionnaire; LSA = Life Space Assessment; WheelCon-P = Wheelchair Use Confidence Scale for powered wheelchair users; WST = Wheelchair Skills Test.
Table 4 presents the percent success rate and agreement for individual skills between the WST-Q and the WST. There were 4 skills that did not meet the threshold of 75% or greater for a clinically significant agreement: ‘operates battery charger’, ‘maneuver sideways’, ‘rolls 2m on soft surface’ and ‘gets over 15cm pothole’. The lack of agreement for the ‘operates battery charger’ skill was, for the most part, a result of the number of participants who scored ‘no part’ on this skill because they did not have their battery charger with them for objective testing, but were able to describe how to perform this skill on the self-report testing. Lack of clinically significant agreement on the other 3 skills appears to be a result of a true discrepancy between perception of skill and actual skill.
Table 4.
Percentage Success Rates and Agreement for Individual Wheelchair Skills Between the WST-Q and WST
| Skill | WST-Q | WST | % Agreement | ||
|---|---|---|---|---|---|
| N | % Success | N | % Success | ||
| 1. Moves Controller away and back | 59 | 98.3 | 57 | 96.4 | 94.4 |
| 2. Turns Controller on and off | 72 | 95.8 | 72 | 95.8 | 97.2 |
| 3. Selects drive modes and speeds | 72 | 100 | 71 | 92.9 | 91.6 |
| 4. Controls Tilt Function | 48 | 97.9 | 48 | 95.8 | 95.8 |
| 5. Control Recline Function | 7 | 85.7 | 6 | 83.3 | 95.8 |
| 6. Disengages and Engages Motors | 71 | 36.6 | 71 | 30.9 | 88.8 |
| 7. Operates Battery Charger | 72 | 61.1 | 34 | 44.1 | 45.8 |
| 8. Rolls Forward 10m | 72 | 100 | 72 | 100 | 100 |
| 9. Rolls forward 10m in 30 sec | 72 | 97.2 | 72 | 100 | 97.2 |
| 10. Rolls backward 5m | 72 | 94.4 | 72 | 98.6 | 93.0 |
| 11. Turns 90° while moving forward | 72 | 100 | 72 | 98.6 | 98.6 |
| 12. Turns 90° while moving backwards | 72 | 91.6 | 72 | 88.8 | 87.5 |
| 13. Turns 180° in place | 72 | 98.6 | 71 | 91.5 | 88.8 |
| 14. Maneuver sideways | 72 | 97.2 | 70 | 62.8 | 62.5 |
| 15. Gets through hinged door | 72 | 65.2 | 71 | 71.8 | 77.7 |
| 16. Reaches 1.5m high object | 72 | 81.9 | 72 | 84.7 | 91.6 |
| 17. Picks object from floor | 72 | 69.4 | 71 | 56.3 | 81.9 |
| 18. Relieves weight from buttocks | 72 | 86.1 | 70 | 82.8 | 84.7 |
| 19. Transfers WC to bench and back | 72 | 54.1 | 72 | 44.4 | 93.0 |
| 20. Rolls 100m | 72 | 100 | 72 | 100 | 100 |
| 21. Avoids moving obstacles | 72 | 94.4 | 71 | 92.9 | 88.8 |
| 22. Ascends 5° incline | 72 | 100 | 71 | 100 | 98.6 |
| 23. Descends 5° incline | 72 | 100 | 71 | 100 | 98.6 |
| 24. Ascends 10° incline | 72 | 91.6 | 71 | 98.5 | 91.6 |
| 25. Descends 10° incline | 72 | 93.0 | 71 | 98.5 | 91.6 |
| 26. Rolls 2m across 5° side slope | 72 | 88.8 | 71 | 94.3 | 84.7 |
| 27. Rolls 2m on soft surface | 72 | 76.3 | 70 | 87.1 | 73.6 |
| 28. Gets over 15cm pothole | 72 | 70.8 | 70 | 52.8 | 65.2 |
| 29. Gets over 2cm threshold | 72 | 91.6 | 70 | 95.7 | 93.0 |
| 30. Ascends 5cm level change | 72 | 62.5 | 71 | 54.9 | 66.6 |
| 31. Descends 5cm level change | 72 | 80.5 | 71 | 81.6 | 80.5 |
| 32. Gets from ground to wheelchair | 72 | 22.2 | 70 | 10.0 | 84.7 |
Note: Skills for which the sample size was less than n=72 were either the result of the participant’s power wheelchair not having the part or function (e.g., no tilt or recline function), participant error, or tester error.
Responsiveness
The SEM and SRD were 5.0 and 6.2 respectively.
DISCUSSION
We accomplished our objective of providing evidence about the measurement properties of the WST-Q. The four hypotheses were verified. We found that the WST-Q has high internal consistency and good test-retest reliability. There is also support for concurrent validity and responsiveness. Participants in this study were older adults with a variety of diagnoses. Generally, this highly experienced sample reported themselves as being quite skilled in power wheelchair use with mean total percentage WST-Q capacity scores of 83.7 (range: 54.8–100) at baseline and 86.3 (range: 54.9–100) at 1 month.
The WST-Q items had a high degree of internal consistency with a Cronbach’s alpha of 0.90. This finding confirms that the WST-Q items are consistent in measuring power wheelchair skills in this sample of older adult power wheelchair users.
The WST-Q retest reliability was good25 with an ICC1,1 of 0.78, but the 95% confidence interval was wide (0.68–0.86). That the second WST-Q was administered after the objective WST may provide a possible explanation for the large confidence interval given the demonstrated training effect induced by simply administering the WST in training studies.35–39 Overall, however, the percent agreement for individual skills between baseline and 1 month indicated that there was a clinically significant agreement between time points.
Our results corroborate our hypotheses regarding the direction of the relationships between the WST-Q and the LSA, WheelCon-P and WST. The correlation between the WST-Q and WST for powered wheelchair users (r=0.65) is less than that for manual wheelchair users (rs=0.89).21 Both the WST-Q and the WST have strengths and limitations. The self-report WST-Q has certain advantages over the objective WST.24 It requires less time, equipment, and space to administer, it may provide a more realistic portrayal of an individual’s skill as it relates specifically to his/her own environment, it is not subject to certain limitations associated with use of objective testing (e.g., failing of a skill due to a technicality, such as crossing over a line), it does not induce a training effect, it avoids a training-to-the-test effect, and there is no risk of injury. It is also sometimes the only option in situations where objective testing is not possible (e.g., telephone follow up, busy outpatient clinic) or is impractical (e.g., individual is on bed rest). Further, the WST-Q can capture information on both capacity and performance.26
The WST-Q also has limitations. As discussed above, it is reliant on the respondent’s ability to understand the questions and provide valid answers. Further, individuals often slightly overestimate their capacity,21 as shown in our results where the mean objective total percentage WST score was slightly lower than the mean total percentage WST-Q score. Finally, details regarding how a skill is performed are not provided, which limits its usefulness as a training guide. This limitation of the WST-Q is the primary strength of the WST in that with objective assessment the tester can observe how a skill is being performed and provide intervention in accordance with the limiting factors (e.g., technique or wheelchair set up). However, there are many factors that may influence skill capacity on any given day, such as motivation or fatigue. Given the merits and limitations of both the WST-Q and WST, there are situations in which one may be more appropriate than the other, but perhaps using them in conjunction with each other provides the most comprehensive representation of an individual’s wheelchair skill.
The magnitude of the relationship between the WST-Q and the WheelCon-P (r=0.48, p<0.001) is similar to that between the WST-Q and the WheelCon for manual wheelchair users (rs=0.58, p<0.001).30 Given that approximately half of the WheelCon-P items are related to negotiating the physical environment, this correlation was expected. Similarly, the magnitude of the relationship between the WST-Q and the LSA (r=0.48, p<0.001) is consistent with the relationship between manual wheelchair skills and wheeled distance (r=0.32, p<0.05)40 and is consistent with previous research findings that the life space mobility of middle-aged and older adults increases after power wheelchair use, especially within their neighborhoods.28
The SEM of 5.0 and the SRD of 6.2 represent the minimal change in WST-Q score that reflects a meaningful change beyond measurement error for a group and an individual respectively. These values meet Smidt et al’s41 standard of 10% or less of the possible score range.
There were limitations to this study, some of which have already been noted. All participants in our sample were experienced power wheelchair users. Although this characteristic was necessary in order to evaluate test-retest reliability, it limited the generalizability of our results. The ICC value might have been higher if there had been a broader range of WST-Q values to evaluate. Second, the sample was one of convenience, a volunteer sample. As such, the characteristics of our sample may differ from that of the general population of older adult power wheelchair users. Third, the 1-month time frame between test and retest may have allowed for learning and/or changes to occur. Although it is unlikely that the participants learned new skills through completing the WST-Q, because a training-to-the-test effect is improbable with a questionnaire, it is possible that participants may have chosen to independently learn about features of their chair for which they were uncertain during the testing. Further, completing an objective WST between the two WST-Q administrations may in itself have had a training effect.34–38 However, the WST-Q difference score of 2.6% between baseline and 1 month does not suggest much of a learning effect. Finally, in this study, the WST-Q was used to assess capacity only and not performance.
Future study is warranted, in particular exploring additional measurement properties, including inter- and intra-rater reliability and construct validity (e.g., relationship between power wheelchair use and previous experience, effect of different types of power wheelchair chairs). As well, capturing both capacity and performance using the WST-Q in future studies will provide a more comprehensive view of power wheelchair skills. The WST-Q will need to continue to evolve and studies like this one will assist this development.
Despite the study limitations and need for further study, this was the first study to investigate the measurement properties of the powered wheelchair version of the WST-Q. These data will help to interpret studies using the WST-Q as an outcome measure and will guide further research in this important area.
CONCLUSIONS
There is support for reliability, validity, and responsiveness for the WST-Q 4.1 with this population of experienced, older adult power wheelchair users. Further study is warranted.
Implications for rehabilitation.
There is evidence of reliability, validity, and responsiveness of the Wheelchair Skills Test Questionnaire (WST-Q) among experienced older adult powered wheelchair users.
The WST-Q can be used to measure powered wheelchair skill, guide intervention, and measure change over time.
Acknowledgments
This study was funded by the Canadian Institutes of Health Research CanWheel team in Wheeled Mobility for Older Adults grant (100925-1). Salary/scholarship funds were provided by: the Canadian Institutes of Health Research (PWR) and the Fonds de recherche du Québec – Santé (FRQS) (FR). We thank Elmira Chan, MEdu, Sarah McCuaig, BA, Jennifer Querques, BPh, Laura Titus, PhD, Fabrizio Farronato, BA, Elise Busilacchi, MSc, Valerie Lemay, MSc, Marie-Pierre Johnson, BSc, Deborah Stewart, PT/OT Asst. Dip, and Lori Kennedy, BSc(Kin) for their assistance with data collection.
Footnotes
Presented in part at: Annual Meeting of RESNA, Baltimore, June 28–July 3, 2012, Baltimore, MD, USA.
Reprints: Not available from the authors.
Clinical Trials Registry Number: NA.
Authors’ financial disclosure: We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated and we certify that all financial and material support for this research are clearly identified in the title page of the manuscript. The authors declare that they have no competing interests.
References
- 1.Brault MW. Americans with disabilities: 2010. U.S. Census Bureau; 2012. [Google Scholar]
- 2.Flagg JF. University at Buffalo: Rehabilitation engineering Research Center on Technology Transfer. 2009. Industry profile on wheeled mobility. [Google Scholar]
- 3.Löfqvist C, Pettersson C, Iwarsson S, Brandt A. Mobility and mobility-related participation outcomes of powered wheelchair and scooter interventions after 4-months and 1-year use. Disabil Rehabil Assist Technol. 2012;7:211–8. doi: 10.3109/17483107.2011.619224. [DOI] [PubMed] [Google Scholar]
- 4.Edwards K, McCluskey A. A survey of adult power wheelchair and scooter users. Disabil Rehabil Assist Technol. 2010;5:411–9. doi: 10.3109/17483101003793412. [DOI] [PubMed] [Google Scholar]
- 5.Frank AO, Neophytou C, Frank J, De Souza L. Electric-powered indoor/outdoor wheelchairs (EPIOCs): users’ views of influence on family, friends and carers. Disabil Rehabil Assist Technol. 2010;5:327–38. doi: 10.3109/17483101003746352. [DOI] [PubMed] [Google Scholar]
- 6.Evans S, Frank AO, Neophytou C, DeSouza L. Older adults’ use of, and satisfaction with, electric powered indoor/outdoor wheelchairs. Age Ageing. 2007;36:431–5. doi: 10.1093/ageing/afm034. [DOI] [PubMed] [Google Scholar]
- 7.Frank AO, Ward J, Orwell NJ, McCullagh C, Belcher M. Introduction of a new NHS electric-powered indoor/outdoor chair (EPIOC) service: benefits, risks and implications for prescribers. Clin Rehabil. 2000;14:665–73. doi: 10.1191/0269215500cr376oa. [DOI] [PubMed] [Google Scholar]
- 8.Xiang H, Chany A-M, Smith GA. Wheelchair related injuries treated in US emergency departments. Injury Prevention. 2006;12:8–11. doi: 10.1136/ip.2005.010033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kirby RL, Ackroyd-Stolarz SA. Wheelchair safety: adverse reports to the United States Food and Drug Administration. Amer J Phys Med Rehabil. 1995;74:308–12. doi: 10.1097/00002060-199507000-00009. [DOI] [PubMed] [Google Scholar]
- 10.Mortenson WB, Hurd Clarke L, Best K. Prescribers’ experiences with powered mobility prescription among older adults. Am J Occup Ther. 2013;67:100–7. doi: 10.5014/ajot.2013.006122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.World Health Organization. Guidelines on the provision of manual wheelchairs in less resourced settings. Geneva: WHO Press; 2008. [PubMed] [Google Scholar]
- 12.Eggers SL, Myaskovsky L, Burkitt KH, Tolerico M, Switzer GE, Fine MJ, Boninger ML. A Preliminary model of wheelchair service delivery. Arch Phys Med Rehabil. 2009;90:1030–8. doi: 10.1016/j.apmr.2008.12.007. [DOI] [PubMed] [Google Scholar]
- 13.Mountain AD, Kirby RL, Eskes GA, Smith C, Duncan H, MacLeod DA, Thompson K. Ability of people with stroke to learn powered wheelchair skills: A pilot study. Arch Phys Med Rehabil. 2010;91:596–601. doi: 10.1016/j.apmr.2009.12.011. [DOI] [PubMed] [Google Scholar]
- 14.Mountain A, Kirby RL, Smith C, Eskes G, Thompson K. Powered wheelchair skills training for persons with stroke: a randomized controlled trial. Am J Phys Med Rehabil. doi: 10.1097/PHM.0000000000000229. (in press) [DOI] [PubMed] [Google Scholar]
- 15.Kirby RL, Smith C, Parker K, MacLeod DA, McAllister M, Rushton PW, Routhier F. [Accessed December 2, 2013];Wheelchair Skills Program General Introduction. Available at: http://www.wheelchairskillsprogram.ca/eng/documents/WSP_General_Introduction_2013-04-17.pdf.
- 16.Kirby RL, Smith C, Parker K, MacLeod DA, McAllister M, Rushton PW, Routhier F. [Accessed December 2, 2013];Wheelchair Skills Test Version 4.1 Manual. Available at: http://www.wheelchairskillsprogram.ca/eng/documents/WST_Manual_version_4.1.60.pdf.
- 17.Kirby RL, Swuste J, Dupuis DJ, MacLeod DA, Monroe R. The Wheelchair Skills Test: a pilot study of a new outcome measure. Arch Phys Med Rehabil. 2002;83:10–8. doi: 10.1053/apmr.2002.26823. [DOI] [PubMed] [Google Scholar]
- 18.Kirby RL, Dupuis DJ, MacPhee AH, et al. The Wheelchair Skills Test (version 2. 4): measurement properties. Arch Phys Med Rehabil. 2004;85:794–804. doi: 10.1016/j.apmr.2003.07.007. [DOI] [PubMed] [Google Scholar]
- 19.Routhier F, Demers L, Kirby RL, Pervieux I, Depa M, DeSerres L, Loiselle F, Dessureault D. Inter-rater and test-retest reliability of the French-Canadian Wheelchair Skills Test (version 3.2): preliminary findings. Proceedings of the Annual Meeting of RESNA; 2007 June 15–19; Phoenix, AZ. [Google Scholar]
- 20.Lindquist NJ, Loudon PE, Magis TF, Rispin JE, Kirby RL, Manns PJ. Reliability of the Wheelchair Skills Test (WST) version 4. 1 for manual wheelchair users. Arch Phys Med Rehabil. 2010;91:1752–7. doi: 10.1016/j.apmr.2010.07.226. [DOI] [PubMed] [Google Scholar]
- 21.Rushton PW, Kirby RL, Miller WC. Manual wheelchair skills: objective testing versus subjective questionnaire. Arch Phys Med Rehabil. 2012;93:2313–8. doi: 10.1016/j.apmr.2012.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Mountain AD, Kirby RL, Smith C. The Wheelchair Skills Test, version 2. 4: validity of an algorithm-based questionnaire version. Arch Phys Med Rehabil. 2004;85:416–23. doi: 10.1016/s0003-9993(03)00427-1. [DOI] [PubMed] [Google Scholar]
- 23.Newton AM, Kirby RL, MacPhee AH, Dupuis DJ, MacLeod DA. Evaluation of manual wheelchair skills: is objective testing necessary or would subjective estimates suffice? Arch Phys Med Rehabil. 2002;83:1295–9. doi: 10.1053/apmr.2002.33071. [DOI] [PubMed] [Google Scholar]
- 24.Kirby RL, Smith C, Parker K, MacLeod DA, McAllister M, Rushton PW, Routhier F. [Accessed December 2, 2013];Wheelchair Skills Test Questionnaire Manual Version 4.2. Available at: http://www.wheelchairskillsprogram.ca/eng/documents/WST-Q_Manual_Version_4.2.4.pdf.
- 25.Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 2. Upper Saddle River, New Jersey: Prentice-hall Inc; 2000. [Google Scholar]
- 26.World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001. [Google Scholar]
- 27.Baker PS, Bodner EV, Allman RM. Measuring life-space mobility in community-dwelling older adults. J Am Geriatr Soc. 2003;51:1610–4. doi: 10.1046/j.1532-5415.2003.51512.x. [DOI] [PubMed] [Google Scholar]
- 28.Auger C, Demers l, Gelinas I, Miller WC, Jutai JW, Noreau L. Life-space mobility of middle-aged and older adults at various stages of usage of power mobility devices. Arch Phys Med Rehabil. 2010;91:765–73. doi: 10.1016/j.apmr.2010.01.018. [DOI] [PubMed] [Google Scholar]
- 29.Rushton PW, Demers L, Miller WC CanWheel Research Team. Measurement properties of the Wheelchair Use Confidence Scale for Power Wheelchair Users (WheelCon-P). Proceedings of the 41st Annual Scientific and Educational Meeting of the Canadian Association on Gerontology; Vancouver, British Columbia. October 18–20, 2012. [Google Scholar]
- 30.Rushton PW, Miller WC, Kirby RL, Eng JJ. Measure for the assessment of confidence with manual wheelchair use (WheelCon-M): reliability and validity. J Rehabil Med. 2013;45:61–7. doi: 10.2340/16501977-1069. [DOI] [PubMed] [Google Scholar]
- 31.Andresen EM. Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil. 2000;81(S2):S15–S20. doi: 10.1053/apmr.2000.20619. [DOI] [PubMed] [Google Scholar]
- 32.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10. [PubMed] [Google Scholar]
- 33.Domholdt E. Physical therapy research: principles and applications. 2. Philadelphia: Saunders; 2000. [Google Scholar]
- 34.Beckerman H, Roebroeck ME, Lankhorst GJ, Becher JG, Bezemer PD, Verbeek AL. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res. 2001;10:571–8. doi: 10.1023/a:1013138911638. [DOI] [PubMed] [Google Scholar]
- 35.Ozturk A, Ucsular FD. Effectiveness of a wheelchair skills training programme for community-living users of manual wheelchairs in Turkey: a randomized controlled trial. Clin Rehabil. 2011;25:416–24. doi: 10.1177/0269215510386979. [DOI] [PubMed] [Google Scholar]
- 36.Best KL, Kirby RL, Smith C, MacLeod DA. Wheelchair skills training for community-based manual wheelchair users: a randomized controlled trial. Arch Phys Med Rehabil. 2005;86:2316–23. doi: 10.1016/j.apmr.2005.07.300. [DOI] [PubMed] [Google Scholar]
- 37.Coolen AL, Kirby RL, Landry J, MacPhee AH, Dupuis D, Smith C, Best KL, MacKenzie DE, MacLeod DA. Wheelchair skills training program for clinicians: a randomized controlled trial with occupational therapy students. Arch Phys Med Rehab. 2004;85:1160–67. doi: 10.1016/j.apmr.2003.10.019. [DOI] [PubMed] [Google Scholar]
- 38.MacPhee AH, Kirby RL, Coolen AL, Smith C. Wheelchair skills training program: a randomized clinical trial of wheelchair users undergoing initial rehabilitation. Arch Phys Med Rehab. 2004;85:41–50. doi: 10.1016/s0003-9993(03)00364-2. [DOI] [PubMed] [Google Scholar]
- 39.Routhier F, Kirby RL, Demers L, Depa M, Thompson K. Efficacy and retention of the French-Canadian version of the Wheelchair Skills Training Program for manual wheelchair users: a randomized controlled trial. Arch Phys Med Rehabil. 2012;93:940–8. doi: 10.1016/j.apmr.2012.01.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Lemay V, Routhier F, Noreau L, Phang SH, Martin Ginis KA. Relationships between wheelchair skills, wheelchair mobility and level of injury in individuals with spinal cord injury. Spinal Cord. 2012;50:37–41. doi: 10.1038/sc.2011.98. [DOI] [PubMed] [Google Scholar]
- 41.Smidt N, van der Windt DA, Assendelft WJ, Mourits AJ, Deville WL, de Winter AF, Bouter LM. Interobserver reproducibility of the Assessment of Severity of Complaints, Grip Strength, and Pressure Pain Threshold in patients with lateral epicondylitis. Arch Phys Med Rehabil. 2002;83:1145–50. doi: 10.1053/apmr.2002.33728. [DOI] [PubMed] [Google Scholar]

