Reliability and Validity of Ratings of Perceived Difficulty During Performance of Static Standing Balance Exercises

Abstract

Background

Standardized instruments for measuring the intensity of balance exercises in clinical environments are lacking.

Objective

The objective of this study was to develop a method for quantifying the perceived intensity of standing balance exercises.

Design

A test-retest study design was used, with repeated evaluations within the same visit and between visits 1 week later.

Methods

Sixty-two participants who were healthy and 18 to 85 years old (with a mean age of 55 years [SD = 20 years]; 50% women) were enrolled. On each of 2 visits, they performed 2 sets of 24 randomized static standing exercises consisting of combinations of the following factors: surface, vision, stance, and head movement. Postural sway was measured with an inertial measurement unit, and ratings of perceived difficulty (RPD) were recorded using numerical and qualitative scales. The RPD scales were validated against the quantitative sway measures using a general linear model approach. The test-retest reliability of the RPD scales was examined using a weighted kappa coefficient.

Results

Both RPD scales were associated with postural sway measures with correlation coefficients > 0.6 for the whole sample. The test-retest reliability of the ratings varied considerably across the different balance exercises, and the highest weighted kappa values occurred for RPD scores on the numerical scale within the second visit, as moderate agreement was achieved in 18 of the 24 exercises.

Limitations

The limitations are that the RPD scales need to be validated for other types of balance exercises and in individuals with balance disorders.

Conclusions

The RPD scores correlated with the magnitude of postural sway, suggesting that they can be used as a proxy measure of perceived intensity of balance exercises.

Targeted balance and vestibular rehabilitation exercises have been shown to improve balance in older adults and people with vestibular disorders.^1–4 Similarly, a number of studies that have investigated interventions to prevent falls have found that balance training can be a potentially important strategy for reducing falls in older adults.^4–7

Balance and vestibular rehabilitation therapy comprises different categories of exercises such as static standing, weight shifting, anticipatory postural adjustments, gait, and eye-head coordination.^8,9 One of the key elements to consider when prescribing any exercise for functional improvement during rehabilitation is the intensity of the exercise. However, there is no standard way for measuring intensity of balance exercises¹⁰ as there is for aerobic (eg, percent of heart rate reserve) and resistance (eg, percent of the 1-repetition maximum) exercises,¹¹ and the documentation of exercise intensity in clinical trials of balance interventions is limited.^10,12,13 During balance and vestibular rehabilitation, physical therapists progress the challenge of standing balance exercises by reducing sensory input (eg, standing on foam or closing eyes), changing the base of support (eg, standing in semitandem stance), and perturbing the balance system (eg, moving the head in yaw or pitch directions). The progression of intensity of balance exercises is usually done based on experience rather than rubrics based on evidence.⁸

To assess the intensity of standing balance exercises, a physical therapist can employ several different methods. Quantitative posturography by use of force platforms and inertial measurement units such as accelerometers are considered the gold standard for measuring control of standing balance^14–17 and thus may represent an indicator of standing balance exercise intensity based on the concept that increased body sway or center of pressure reflects increased balance challenge. However, most clinics do not have the ability to measure or interpret postural sway data. Thus, many clinicians may use methods such as visually observing the amount of sway, timing the duration of maintaining stance, or asking individuals for a self-report of the difficulty of exercises.¹² However, ratings of visual observation of sway amount such as “normal sway” or “increased sway” may lack precision, and evidence of interrater reliability has not been established. The timing of standing balance exercises can be subject to a ceiling effect if the clinician limits testing to 30 seconds. Validated self-report measures that specifically assess balance intensity are not available, which presents a barrier to being able to prescribe the initial intensity and progress the intensity of balance exercises. In other rehabilitation settings, ratings of perceived exertion are commonly used to monitor relative intensity of aerobic or resistance exercises (eg, Borg Rating of Perceived Exertion¹⁸ and OMNI Resistance Exercise Scale^19,20). Thus, the motivation for this research was to determine if a similar rating method could be developed for standing balance exercises.

As an initial step in developing self-reported intensity ratings across the realm of balance exercises, we investigated the psychometric properties of rating of perceived difficulty (RPD) scales for static standing balance exercises because they are commonly used and quantitative methods of assessment are well established. Consequently, we validated 2 RPD scales by comparing them with quantitative sway measures taken during performance of standing balance activities. In addition, we estimated the test-retest reliability of the RPD scales within and between 2 visits occurring 1 week apart. The availability of an RPD scale could complement the tools physical therapists currently use for assessing intensity for standing balance exercises and assist in customizing prescription and progression.

Methods

Participants

Sixty-two people who were healthy, independently participating in daily activities, and 18 to 85 years old (31 women and 31 men; mean age = 55 years [SD = 20 years]) participated in this study. The participants of this study were recruited through advertisements posted at the University of Pittsburgh’s campus and through communication with people registered in the Clinical and Translational Science Institute at the University of Pittsburgh. Study participants were distributed into 4 groups—young (18–44 years; n = 17), middle-aged (45–59 years; n = 15), old (60–74 years; n = 15), and very old (75–85 years; n = 15)—to obtain adequate sampling across the adult population and examine if there was an association between age and the properties of the instrument.

Participants were excluded if they had an inability to stand for 3 minutes without rest; clinically significant somatosensory loss (unable to complete the Romberg test for 30 seconds and unable to feel a pressure of a 4.31-g monofilament applied on the dorsum of the foot and the medial side of the foot below the medial malleolus with eyes closed); visual acuity worse than 20/40 using a standard eye chart; a peripheral vestibular disorder (positive head thrust test); a diagnosis of benign paroxysmal positional vertigo (positive Dix-Hallpike test or positive roll test); history of neurological or orthopedic disorders; excessive weight (body mass index of > 35); cognitive impairment (≤ 25 points on the Montreal Cognitive Assessment); history of falling 2 times or more within the last 12 months during activities of daily living; were pregnant; or used an assistive device for ambulation. This study was approved by the Institutional Review Board of the University of Pittsburgh, and all participants provided informed consent prior to participating in the study.

Experimental Procedure

Eligible participants who met the study criteria completed the Activities-Specific Balance Confidence Scale questionnaire²¹ and the Functional Gait Assessment,²² and their gait speed²³ was recorded prior to the experiment to describe the participant characteristics. Gait speed was measured over 6 m. Participants were instructed to walk at a comfortable speed for 10 m, and their average speed was timed during the middle 6 m to avoid the effects of gait initiation and termination. Gait speed values were collected over 3 trials, and the average of the 3 trials was calculated.

Participants were tested during 2 experimental visits, 1 week apart, based on previous balance performance reliability studies and typical vestibular rehabilitation practice patterns.^22,24–27 During each experimental visit, participants performed 2 sets of 24 randomized static standing balance exercises, which were a full-factorial design of the following different factors: surface (firm and foam); vision (eyes open and eyes closed); base of support (feet apart and semitandem); and head movements (head still, yaw, and pitch). Participants stood barefoot to avoid the confounding effect of wearing different shoes. The foam surface was an AIREX Balance Pad (S34-55; AIREX, Sins, Switzerland).²⁸ During the feet apart condition, participants stood with their heel centers 0.17 m apart with an angle of 14 degrees between the long axes of the feet.²⁹ For the semitandem stance position, participants stood with the nondominant front foot touching the medial side of the dominant back foot by a half of a foot length. The dominant foot was determined by asking the participants about the foot that they would use to kick a ball.³⁰ During the eyes closed conditions, participants wore opaque goggles. During yaw and pitch conditions, participants moved their head at a frequency of 1 Hz in synchronization with the beat of a metronome³¹ within a range of ±45 degrees in the yaw direction³² and ±30 degrees in the pitch direction with respect to a neutral head position. Before they started the experiment, participants practiced moving their head in yaw and pitch directions with a laser light attached to the head. However, the laser light was not used during the experiment.

Exercises were performed using a separate randomization scheme for each participant that was software-generated. Participants were instructed to stand as stable as possible with arms at their side during all trials for 35 seconds (Fig. 1). Data collection was stopped if participants lost their balance according to the following failure criteria: stepped out of position, changed position of feet or arms from the starting position, and/or touched something for support. Participants were asked to repeat failed trials once in each set if they lost their balance before completing 25 seconds. If an exercise could not be completed during the first set of each visit, it was not attempted during the second set. Participants were guarded by a physical therapist during all exercises to prevent falling and wore a loosely tethered safety harness, which was attached to an anchor point in the ceiling. There was a seated rest break for 1 minute after every 3 exercises to avoid fatigue.

Figure 1.

Experimental setup. Participant is standing on a foam pad with the feet apart. Rating scales are shown to the left of the participant. The orange inertial measurement unit (IMU; Xsens Technologies BV, Enschede, the Netherlands) is attached to the waist of the participant to measure postural sway (tilt displacement, tilt velocity, and linear acceleration).

The RPD scales used in this study were conceptually similar to rating of perceived exertion scales commonly used to assess perceived cardiovascular workload. However, the term “difficulty” was used in place of “exertion” because we anticipated that it would be more easily understood by users in the context of a balance task. After each exercise, participants provided separate RPD scores using 2 different scales. The first was a numerical scale (0–10) with verbal anchors based on the OMNI Perceived Exertion Scale (RPD-numerical scale),^19,33 where 0 indicated that the exercise was extremely easy and 10 indicated that the exercise was extremely hard (Fig. 2). The second scale was qualitative (RPD-qualitative scale) (Fig. 2), modeled after the work of Espy et al,^34,35 who developed a 10-point “Rate of Perceived Stability” color-coded scale with verbal anchors ranging from “Completely Stable to “About to Fall.” The RPD-qualitative scale had 5 color-coded levels (A–E) and verbal anchors, where A was labeled with the statement “I feel completely steady,” and E was labeled with the statement “I lost my balance.” In the statistical analysis, letters from the RPD-qualitative scale were transformed to numbers as follows: A = 1, B = 2, C = 3, D = 4, and E = 5. Prior to this investigation, the RPD-qualitative scale was piloted in several research studies.

Figure 2.

Rating of perceived difficulty scales. (A) Numerical scale based on the OMNI Perceived Exertion Scale.^19,33 (B) Qualitative scale modeled after the work of Espy et al.^34,35

Before starting the experiment, both scales were explained to the participants. They were instructed to report after each exercise a number from the RPD-numerical scale and a letter from the RPD-qualitative scale that indicated the difficulty of maintaining their balance during that exercise. During the experiment, the scales were placed on the side wall so that participants could easily view them. The numerical scores were provided first, followed by the qualitative ratings during the first set of exercises, and vice versa during the second set of exercises for each visit. A rating was provided for each trial attempted, even if the trial was not completed. At the end of the experiment, participants were verbally asked which of the 2 scales (numerical or qualitative) they liked the best.

Postural Sway Measures

During the performance of the exercises, an inertial measurement unit (Xsens Technologies BV, Enschede, the Netherlands)^36,37 was mounted on each participant’s lower back at the level of iliac crest (L4) (Fig. 1). The inertial measurement unit was used to measure trunk angular displacement and velocity in the pitch and roll directions, and linear acceleration in the anteroposterior (AP) and mediolateral (ML) directions at sampling rate of 100 Hz. Sway measures were recorded during all trials for 35 seconds, and the first 5 seconds of data were removed to eliminate any effects associated with the initial establishment of balance.^38,39 Summary measures of trunk sway were calculated from the remaining 30-second time series. The data were low-pass filtered using a second-order Butterworth filter with a cut-off frequency of 3 Hz.^40,41 During the analysis, each trial was plotted individually and inspected visually using MATLAB software (The MathWorks, Inc, Natick, MA, USA) to make sure that there were no extraneous movements. Postural sway data from incomplete trials were treated as missing data.

The root-mean-square (RMS) of the trunk angular displacement and velocity in the pitch and roll directions, and linear acceleration in the AP and ML directions were calculated and used in the analyses. The RMS was calculated as follows:

where a is the instantaneous sway value with the mean value subtracted, i is an individual data sample, and n is the total number of samples.

Data Analyses

Participants’ demographic characteristics were compared between groups using a 1-way analysis of variance for dependent variables that were continuous and normally distributed, and post hoc comparisons were conducted to evaluate pairwise differences among the groups. The Sidak approach was used to control for a type I error.⁴² The Kruskal-Wallis test was used with dependent variables that were continuous but not normally distributed, and the Dunn procedure⁴³ was used for pairwise comparisons with a Bonferroni correction for multiple comparisons.

To assess the concurrent validity of the RPD scales, the relationship between each RPD scale and each postural sway variable was assessed using a general linear model method that accounted for correlated data (the scores of RPD and postural sway measures) within each participant.⁴⁴ Because there were as many as 24 observations per participant (ie, 1 observation for each exercise), the regression treated participant as a fixed effect to remove the variation due to participants. For each exercise, the mean of the RPD values and postural sway values from all of the 4 trials were used in the model. The RPD scale was the outcome variable, participant was a fixed effect, and the postural measure was entered as a covariate (RPD = intercept + participant + postural measure + error). From the analysis of variance table, the correlation between the RPD and the postural sway measure magnitude while controlling for the interparticipant variability, was computed by the following formula:

The direction of the correlation is given by the sign of the slope of the regression coefficient between the RPD and postural measure. Correlation coefficients were calculated for each RPD scale and all of the postural variables listed previously. In addition, we calculated the correlation coefficients for all participants and each group of participants.⁴⁴

After visually examining the data, we observed that the association between the scores of the RPD scales and the postural sway measures was curvilinear, especially for the old and very old groups. Consequently, we tested both linear and semilogarithmic relationships between the RPD scales and postural sway measures. Expressing the postural sway values on a logarithmic scale (base 10) has been suggested by Schieppati et al when looking at the relationship between body sway and subjective report of steadiness.⁴⁵ We determined that the logarithmic relationship provided a slightly better fit (increase in correlation coefficient by ~ 0.02 on average) between the scores of the RPD scales and the postural sway measures.

The RPD scales were also validated against the theoretical construct that the perceived difficulty would be greater in the conditions of foam surface versus firm surface, eyes closed versus eyes open, semitandem stance versus feet apart, and yaw and pitch head movement versus head still. Consequently, we examined the relationship between the balance exercise factors (surface, vision, base of support, and head movement) and RPD scores using the nonparametric Friedman test for the 3-level head movement factor and the Wilcoxon signed rank test for the other, 2-level factors.

To explore the test-retest agreement (reliability) of both RPD scales, a weighted kappa (linear weight) was used because the data were ordinal. For each of the 24 exercises, test-retest agreement was assessed within the 2 sets of exercises for each visit and between the first set of exercises for both visits. Missing data due to trials that were not performed remained as missing data during the statistical analysis. Point estimates and 95% confidence intervals (CIs) of the weighted kappa were calculated for each exercise. The lower bound of the 95% CI was used to interpret the strength of agreement using criteria suggested by Landis and Koch⁴⁶: excellent agreement = 0.81 to 1; substantial agreement = 0.61 to 0.80; moderate agreement = 0.41 to 0.60; fair agreement = 0.21 to 0.40; and poor agreement = 0.01 to 0.20.

Role of the Funding Source

This study was supported by grants from the National Institutes of Health (5-R21-DC-012410-02, UL1TR001857). The funder played no role in the design, conduct, or reporting of this study.

Results

Of the 72 people who underwent onsite screening, 62 completed the study and were assigned to 1 of 4 groups. The 10 people (7 men) who did not complete the study had a mean age of 64 years (SD = 14 years). Eight people were excluded because they did not pass the inclusion criteria (4 did not pass the cognitive test, 3 did not pass the monofilament test, and 1 did not pass the roll test), and 2 people did not come back for the experimental visits for unknown reasons. Table 1 shows the demographic and clinical characteristics of the participants. Overall, the data showed values consistent with age-normed populations who are healthy. The very old group had a significantly lower gait speed than the young (P = .01) and middle-aged (P = .007) groups. The young group had a lower body mass index than the old (P = .001) and very old (P = .009) groups. The very old group had lower Activities-Specific Balance Confidence Scale scores than the young group (P = .023) and lower Functional Gait Assessment scores than all other groups (P = .003).

Table 1.

Demographic and Clinical Characteristics of Participants

	Value for Participants in the Following Group
Characteristic	All participants (18–85 y Old) (N = 62)	Young (18–44 y Old) (n = 17)	Middle-aged (45–59 y Old) (n = 15)	Old (60–74 y Old) (n = 15)	Very old (75–85 y Old) (n = 15)
Age, y, mean (SD)	55 (20)	28 (8)	53 (4)	67 (4)	79 (3)
No. (%) women	31 (50)	9 (53)	8 (53)	7 (47)	7 (47)
Body mass index, kg/m2, median (range)	26.3 (15.5–35.8)	21.8 (18.1–33.5)	27.5 (18.1–32.1)	29.9 (15.5–34.8)	27.8 (19.9–35.8)
Semmes–Weinstein monofilament handle marking, median (range)	4.08 (2.83–4.31)	3.84 (2.83–4.08)	4.08 (2.83–4.31)	4.08 (3.61–4.17)	4.17 (3.61–4.31)
Montreal Cognitive Assessment score, median (range)	29 (26–30)	29 (26–30)	28 (26–30)	29 (26–30)	28 (26–30)
Activities–Specific Balance Confidence Scale score, median (range)	97 (81–100)	99 (89–100)	94 (83–100)	98 (81–100)	91 (88–99)
Gait speed, m/s, mean (SD)	1.30 (0.20)	1.38 (0.20)	1.39 (0.21)	1.26 (0.19)	1.16 (0.12)
Functional Gait Assessment score, median (range)	28 (19–30)	29 (27–30)	29 (23–30)	28 (19–30)	24 (19–29)

In the examination of the relationship between RPD scores and experimental conditions, both RPD scores were observed to be greater on a foam surface than on a firm surface (Z = 6.85; P < .001), with eyes closed than with eyes open (Z = 6.85; P < .001), with a semitandem stance than with feet apart (Z = 6.85; P < .001), and with yaw and pitch head movements than with head still (Χ² = 93; P < .001). Reviewing the noncompletion rate of exercises in each age group (Suppl. Tab. 1, available at http://academic.oup.com/ptj) revealed that most participants in the young group could complete all exercises except for the exercise performed on a foam surface with eyes closed in a semitandem stance with yaw head movements (exercise 23 in Suppl. Tab. 1). The majority of the middle-aged group could complete all exercises except the ones performed on a foam surface with eyes closed in a semitandem stance with yaw and pitch head movements (exercises 23 and 24); noncompletion rates were 66.7%, and 65%, respectively. In addition to these exercises, participants in the older adult group had more difficulty completing exercises on a foam surface with eyes open in a semitandem stance while performing yaw and pitch head movements (exercises 17 and 18). Older adults had higher rates (13.3%–73.3%) of noncompletion than middle-age adults. The number of exercises that participants who were 75 through 85 years old could not perform was higher than that of other groups and included the exercises noted above in addition to several other exercises (such as exercises 11, 12, 21, and 22); noncompletion rates ranged from 11.7% to 96.7%. Mean and SD of the RPD scores for each exercise are detailed in Supplementary Table 2 (available at https://academic.oup.com/ptj).

A general linear model was used to assess the association between the scores of the RPD scales and the logarithm of the postural sway measures across the balance exercises while accounting for repeated observations.⁴⁴ Missing postural sway data due to exercises that could not be completed accounted for < 5% of the possible data points in the model (ie, 64 missing data points out of 1488). Figure 3 illustrates the linear relationship between average RPD score and average value of the RMS roll velocity, demonstrating the concurrent validity of the RPD scales. The positive correlations ranged from 0.48 to 0.88 between the scores of the RPD scales and the logarithm of the postural sway measures with few exceptions (all Ps < .001 relative to the null hypothesis that the correlation = 0) (Tab. 2). For all age groups combined, the correlation between the RPD-numerical scale and roll angular position was greater than the correlation between the RPD-numerical scale and pitch angular position (z = 1.98; P = .024). Similarly, greater correlations occurred between the RPD-numerical scale and roll velocity than with pitch velocity (z = 2.20; P = .014), the RPD-qualitative scale and roll position than with pitch position (z = 1.81; P = .035), and finally the RPD-qualitative scale and roll velocity than with pitch velocity (z = 1.89; P = .029).

Figure 3.

Relationship between the average of the rating of perceived difficulty (RPD) scales and the average of the root-mean-square (RMS) roll velocity across all participants. Each point represents 1 of the 24 exercises. (A) RPD numerical scale (0–10). (B) RPD qualitative scale (1–5).

Table 2.

Correlation Coefficients^a for Relationship Between RPD Scales^b and Logarithm^c of RMS of Postural Sway Measures for Each Study Group^d

		Correlation Coefficients for the Postural Sway Measures
Scale	Group (Age, y)	RMS of Pitch Displacement	RMS of Roll Displacement	RMS of Pitch Velocity	RMS of Roll Velocity	RMS of AP Acceleration	RMS of ML Acceleration
Numerical	Young (18–44)	0.66	0.83	0.54	0.80	0.69	0.82
	Middle-aged (45–59)	0.77	0.87	0.63	0.81	0.76	0.85
	Old (60–74)	0.74	0.87	0.58	0.76	0.69	0.81
	Very old (75–85)	0.72	0.86	0.68	0.82	0.72	0.82
	All (18–85)	0.73	0.86	0.60	0.80	0.71	0.82
Qualitative	Young (18–44)	0.63	0.77	0.48	0.74	0.63	0.75
	Middle–aged (45–59)	0.74	0.84	0.58	0.78	0.73	0.81
	Old (60–74)	0.74	0.88	0.56	0.76	0.69	0.81
	Very old (75–85)	0.73	0.84	0.68	0.79	0.72	0.79
	All (18–85)	0.71	0.84	0.57	0.76	0.69	0.79

^{a a}Correlation = √[sum of squares for postural measure/(sum of squares for postural measure + sum of squares for residual)].⁴⁴

^{b b}Numerical and qualitative.

^{c c}Base 10.

^{d d}All correlation coefficients were significant at P < .001. AP = anteroposterior; ML = mediolateral; RMS = root–mean–square; RPD = ratings of perceived difficulty.

A weighted kappa (linear weight) was used to explore the test-retest agreement of RPD by participants who were healthy of 24 static stance balance exercises within and between 2 visits occurring 1 week apart. Point estimates of the weighted kappa scores for the RPD-numerical scale ranged from 0.32 to 0.68 for all 24 exercises performed over 2 times within the first visit and within the second visit (Tab. 2). Using the lower bound of the 95% CI to interpret the strength of the agreement, the RPD-numerical scale achieved moderate agreement for 4 of the 24 exercises within the first visit and 18 of the 24 exercises performed on the second visit. No exercises produced moderate agreement between visits 1 and 2. Fair agreement was reached for more than 20 exercises within each visit and between visits.

Point estimates of weighted kappa scores for the RPD-qualitative scale (Tab. 3) were fair to substantial (0.21–0.74) within the first visit and the second visit except for exercise 1 (firm surface, eyes open, feet apart, and head still). Between visits 1 and 2, weighted kappa scores for the RPD-qualitative scale were fair to substantial (0.28–0.65) except for exercise 1 and exercise 3 (firm surface, eyes open, feet apart, and pitch head movement). Again, using the lower bound of the 95% CI, we observed that moderate agreement was reached for the RPD-qualitative scale ratings on no more than 6 of the balance exercises within and between visits. Fair agreement of RPD-qualitative scale scores was reached between 13 and 18 exercises within each visit and between visits. Forty-one of the 62 participants (66.1%) reported that they liked the RPD-qualitative scale more than the RPD-numerical scale, with many of the participants pointing out that the statements in the RPD-qualitative scale described how they felt.

Table 3.

Linearly Weighted Kappa Coefficients for RPD Scales for Each of 24 Exercises Within Visits 1 and 2 and Between Visits 1 and 2^a

					Kappa Coefficient (95% CI)
					Numerical Scale			Qualitative Scale
					Within Visit		Between Visits	Within Visit		Between Visits
Exercise No.	Surface	Vision	Base of Support	Head Movement	Visit 1, Sets 1 and 2	Visit 2, Sets 1 and 2	1 and 2, Set 1	Visit 1, Sets 1 and 2	Visit 2, Sets 1 and 2	1 and 2, Set 1
1	Firm	EO	FA	Still	0.60 (0.43 to 0.78)	0.58 (0.42 to 0.74)	0.43 (0.24 to 0.63)	−0.03 (−0.07 to 0.00)	0.38 (−0.18 to 0.93)	−0.04 (−0.08 to 0.00)
2	Firm	EO	FA	Yaw	0.57 (0.45 to 0.69)	0.56 (0.40 to 0.73)	0.40 (0.26 to 0.55)	0.64 (0.27 to 1.02)	0.46 (0.14 to 0.78)	0.28 (−0.08 to 0.64)
3	Firm	EO	FA	Pitch	0.50 (0.32 to 0.69)	0.56 (0.41 to 0.71)	0.46 (0.29 to 0.63)	0.21 (−0.07 to 0.50)	0.52 (0.18 to 0.85)	0.14 (−0.14 to 0.43)
4	Firm	EO	ST	Still	0.43 (0.28 to 0.59)	0.55 (0.41 to 0.68)	0.47 (0.31 to 0.62)	0.43 (0.21 to 0.65)	0.47 (0.30 to 0.63)	0.51 (0.36 to 0.66)
5	Firm	EO	ST	Yaw	0.48 (0.33 to 0.63)	0.62 (0.52 to 0.73)	0.41 (0.28 to 0.55)	0.65 (0.50 to 0.79)	0.55 (0.40 to 0.69)	0.44 (0.29 to 0.58)
6	Firm	EO	ST	Pitch	0.52 (0.38 to 0.67)	0.54 (0.40 to 0.67)	0.48 (0.35 to 0.61)	0.55 (0.38 to 0.73)	0.47 (0.32 to 0.63)	0.61 (0.45 to 0.76)
7	Firm	EC	FA	Still	0.56 (0.40 to 0.71)	0.68 (0.55 to 0.81)	0.54 (0.36 to 0.72)	0.42 (0.08 to 0.76)	0.47 (0.26 to 0.68)	0.54 (0.24 to 0.85)
8	Firm	EC	FA	Yaw	0.48 (0.34 to 0.62)	0.60 (0.46 to 0.75)	0.42 (0.28 to 0.57)	0.33 (0.02 to 0.64)	0.74 (0.52 to 0.97)	0.65 (0.42 to 0.88)
9	Firm	EC	FA	Pitch	0.54 (0.39 to 0.69)	0.58 (0.46 to 0.70)	0.37 (0.24 to 0.50)	0.31 (−0.05 to 0.67)	0.48 (0.27 to 0.69)	0.40 (0.18 to 0.62)
10	Firm	EC	ST	Still	0.34 (0.19 to 0.49)	0.56 (0.44 to 0.68)	0.40 (0.26 to 0.54)	0.35 (0.16 to 0.54)	0.44 (0.29 to 0.59)	0.36 (0.18 to 0.53)
11	Firm	EC	ST	Yaw	0.45 (0.31 to 0.59)	0.43 (0.29 to 0.57)	0.49 (0.37 to 0.62)	0.38 (0.21 to 0.55)	0.32 (0.15 to 0.48)	0.49 (0.33 to 0.64)
12	Firm	EC	ST	Pitch	0.51 (0.39 to 0.63)	0.56 (0.42 to 0.69)	0.39 (0.24 to 0.54)	0.42 (0.24 to 0.59)	0.55 (0.39 to 0.71)	0.38 (0.20 to 0.55)
13	Foam	EO	FA	Still	0.43 (0.27 to 0.58)	0.53 (0.41 to 0.64)	0.35 (0.20 to 0.51)	0.49 (0.26 to 0.71)	0.59 (0.39 to 0.78)	0.39 (0.19 to 0.60)
14	Foam	EO	FA	Yaw	0.45 (0.30 to 0.61)	0.53 (0.39 to 0.68)	0.37 (0.20 to 0.55)	0.56 (0.41 to 0.71)	0.50 (0.34 to 0.66)	0.34 (0.16 to 0.52)
15	Foam	EO	FA	Pitch	0.32 (0.17 to 0.48)	0.48 (0.34 to 0.63)	0.34 (0.18 to 0.50)	0.38 (0.19 to 0.56)	0.51 (0.35 to 0.67)	0.42 (0.22 to 0.62)
16	Foam	EO	ST	Still	0.39 (0.24 to 0.54)	0.57 (0.45 to 0.69)	0.36 (0.21 to 0.50)	0.39 (0.20 to 0.58)	0.49 (0.33 to 0.65)	0.40 (0.22 to 0.58)
17	Foam	EO	ST	Yaw	0.47 (0.33 to 0.61)	0.56 (0.45 to 0.68)	0.50 (0.36 to 0.64)	0.39 (0.23 to 0.55)	0.58 (0.44 to 0.73)	0.46 (0.29 to 0.63)
18	Foam	EO	ST	Pitch	0.57 (0.43 to 0.72)	0.41 (0.26 to 0.56)	0.35 (0.23 to 0.48)	0.38 (0.22 to 0.53)	0.39 (0.22 to 0.57)	0.49 (0.35 to 0.64)
19	Foam	EC	FA	Still	0.44 (0.30 to 0.59)	0.55 (0.42 to 0.68)	0.39 (0.24 to 0.55)	0.34 (0.16 to 0.52)	0.60 (0.43 to 0.76)	0.26 (0.06 to 0.45)
20	Foam	EC	FA	Yaw	0.49 (0.35 to 0.64)	0.53 (0.43 to 0.64)	0.36 (0.20 to 0.51)	0.37 (0.18 to 0.56)	0.55 (0.41 to 0.70)	0.46 (0.29 to 0.63)
21	Foam	EC	FA	Pitch	0.47 (0.35 to 0.59)	0.50 (0.36 to 0.64)	0.42 (0.28 to 0.57)	0.45 (0.30 to 0.61)	0.52 (0.36 to 0.68)	0.43 (0.26 to 0.61)
22	Foam	EC	ST	Still	0.43 (0.29 to 0.57)	0.59 (0.47 to 0.71)	0.45 (0.33 to 0.57)	0.38 (0.22 to 0.55)	0.58 (0.43 to 0.73)	0.46 (0.32 to 0.60)
23b	Foam	EC	ST	Yaw	0.37 (0.11 to 0.62)	0.49 (0.29 to 0.69)	0.42 (0.24 to 0.59)	0.42 (0.13 to 0.72)	0.38 (0.13 to 0.63)	0.43 (0.25 to 0.62)
24b	Foam	EC	ST	Pitch	0.39 (0.19 to 0.60)	0.58 (0.42 to 0.74)	0.48 (0.34 to 0.62)	0.28 (0.01 to 0.54)	0.36 (0.12 to 0.59)	0.44 (0.26 to 0.62)

^aValues in bold type were considered to have at least moderate agreement on the basis of the lower bound for the estimate. EC = eyes closed; EO = eyes open; FA = feet apart; RPD = ratings of perceived difficulty; ST = semitandem.

^bBecause of the number of incomplete trials in the old and very old age groups, the kappa coefficients for exercises 23 and 24 should not be generalized for all age groups.

Discussion

The first objective of the study was to validate the RPD-scales with quantitative postural sway measurements of angular position, angular velocity, and linear acceleration. The results demonstrated fairly robust associations between the RPD scales (numerical and qualitative) and postural sway measures on a population average basis (Fig. 3) and age group basis (Tab. 2), indicating that participant perception of difficulty in performing a balance exercise is related to the amount of body sway. Furthermore, the magnitude of the ratings was larger compared with standing on a firm surface with eyes open and feet apart with head still by increasing the compliance of the surface when using foam, by reducing visual input by closing eyes, by reducing the base of support, and by increasing vestibular stimulation when moving the head. Previously, Schieppati et al investigated the relationship between center of pressure and participants’ “feeling of the quality of steadiness” rated on a scale from 0 (worst) to 10 (best).⁴⁵ However, only 4 balance exercises (eyes open and eyes closed, during feet apart and feet together stances) were studied.⁴⁵ They discovered that a logarithmic function described the relationship between the center of pressure sway area and steadiness. The nonlinear relationship between perception of balance and sway is consistent with many sensory psychophysical experiments.⁴⁷ We also observed slightly higher correlation coefficients by using the logarithm of the postural sway compared with the untransformed sway values. In particular, older adults had a larger increase in the magnitude of the correlation coefficients by using the log-transformed values compared with the other groups, indicating greater nonlinearity in the relationship of perceived difficulty to postural sway in older adults.

The correlation between RPD scales and postural sway was greater for the roll angular position and velocity than for the pitch angular position and velocity. This finding suggests that participants may have based their RPD more on their perception of sway in the ML direction. The fact that half of the exercises required a limited base of support in the ML direction (ie, semitandem stance) compared with the AP direction contributed to a greater amount of sway in the ML direction relative to the size of the base of support and may explain the stronger relationship between the sway measures in the ML (and roll) direction and the RPD.

Another objective of the study was to estimate the test-retest reliability (agreement) of the RPD scales. The lower bound of the 95% CIs of the RPD-numerical scale scores achieved a moderate agreement for 18 of the exercises performed during visit 2, which was more than the moderate agreement for 4 of the exercises performed during visit 1. The improvement observed for the within-visit agreement from visit 1 to visit 2 suggests that users of the RPD-numerical scale calibrate their ratings with repeated use. Thus, for use within a clinical setting, we recommend use of the RPD scale during every visit as part of a standard routine for assessing intensity.

The weighted kappa coefficients produced a smaller number of exercises with moderate agreement for the RPD-qualitative scale than for the RPD-numerical scale on a within-visit basis, and this finding may be explained by the number of levels that each scale possesses. The RPD-qualitative scale is considered a short scale, with 5 levels, compared with the RPD-numerical scale, which has 11 levels. Compared with longer scales, shorter scales tend to have a prevalence effect that associates inversely with the magnitude of kappa coefficients.⁴⁸ The prevalence effect is present when there is a large difference in the proportion of the ratings between the different levels of the classification.⁴⁸ For instance, in a situation where raters choose between 2 classifying cases like positive or negative or even more cases like hard, medium, and easy, a prevalence effect exists when the proportion of agreements on 1 case is high compared with other choices. With an increase in the prevalence effect, the chance agreement will increase, but the kappa coefficient will decrease accordingly. Because there are fewer levels for the RPD-qualitative scale, it is more likely to have a greater prevalence of responses in any 1 of the 5 levels, thus leading to greater likelihood of chance agreement.

Overall, the weighted kappa values were lower than reliability agreement values established for rating of perceived exertion scales related to aerobic and resistance exercise performance. Lamb et al investigated the reliability of the Borg Rating of Perceived Exertion (6–20) during aerobic exercises (progressive treadmill exercise) and found intraclass correlation coefficients (r) of 0.82 to 0.75.⁴⁹ Another study compared the reliability of the Borg Rating of Perceived Exertion (6–20) and the OMNI Perceived Exertion Scale (0–10) and found that both scales were reliable. However, the OMNI Perceived Exertion Scale had higher reliability than the Borg Rating of Perceived Exertion (0.95 and 0.78, respectively).⁵⁰ The lower agreement values for self-reported balance measures than for aerobic and resistance exercises suggest that the perception of standing balance performance may be more complex or more variable than cardiovascular or muscular effort. Furthermore, the standing balance exercises encompassed a wide range of conditions, which was reflected in the variability of kappa values across exercises. In addition, it is possible that the RPD scales that we used require better descriptors or granularity and need additional refinement to optimize their reliability.

This study had several additional limitations. The experimental visits in this study lasted for 1 hour 45 minutes on average, which may have caused fatigue, especially for older adults who required more time for breaks. Because of the length of time of approximately 1 hour between sets when ratings were provided, it is possible that ratings were not completely independent. Nonindependent ratings have been shown to inflate the kappa coefficients; participants’ recollections of their ratings for the first repetition may have influenced their ratings for the second repetition.⁴⁸ However, because of the large number of exercises included in our study and the fact that the trials were randomized, it would have been difficult for participants to recall previous ratings. Randomizing the testing conditions within sets and between visits was attempted to eliminate the order effect due to practice or fatigue. Although this study established reliability and validity of the RPD in a population of adults who were healthy, the properties should be investigated in populations who would benefit from balance rehabilitation, including individuals with vestibular disorders, older adults, and people with other neurological disorders. The standing balance exercises investigated in this study represent a small fraction of the possible types of exercises within a balance rehabilitation program as well as a subset of the possible combination of factors used for standing balance exercises (eg, other base of support conditions including feet together, tandem stance, and single leg stance). Future investigations could explore people’s perceived difficulty ratings during other types of exercises (eg, modified center of gravity, weight shifting, gait) and/or other standing balance conditions. Although the relationship between perceived difficulty and postural sway appears valid for standing balance exercises and has theoretical rationale, we will need to explore which instrumented measures are most appropriate to demonstrate validity of the rating for other balance rehabilitation exercises.

The main strength of the rating scales is their ease of use and interpretation in clinical settings compared with technology-based techniques for assessing postural sway. Although the RPD-qualitative scale was liked by more participants, we believe that the greater agreement and granularity of the RPD-numerical scale compared with the RPD-qualitative scale makes the RPD-numerical scale a slightly better choice for use as a proxy measure of intensity of standing balance exercises. The use of an RPD scale at each visit may facilitate better exercise progression algorithms used in practice and research. Additionally, RPD can be used for home-based balance exercise training to provide feedback to the physical therapist of how well the exercises met a targeted intensity.

Author Contributions and Acknowledgments

Concept/idea/research design: S.F. Alsubaie, S.L. Whitney, J.M. Furman, G.F. Marchetti, K.H. Sienko, B.N. Klatt, P.J. Sparto

Writing: S.F. Alsubaie, S.L. Whitney, J.M. Furman, G.F. Marchetti, K.H. Sienko, P.J. Sparto

Data collection: S.F. Alsubaie

Data analysis: S.F. Alsubaie, J.M. Furman, G.F. Marchetti, P.J. Sparto

Project management: S.F. Alsubaie, P.J. Sparto

Providing participants: S.F. Alsubaie

Providing facilities/equipment: J.M. Furman, P.J. Sparto

Consultation (including review of manuscript before submitting): S.F. Alsubaie, S.L. Whitney, J.M. Furman, G.F. Marchetti,K.H. Sienko, B.N. Klatt, P.J. Sparto

The authors acknowledge the assistance of the following people in helping with data collection: Dr Abdulaziz Alkathiry, Anita Lieb, Susan Strelinski, Dr Chia-Cheng Lin, Dr. Mohammed Alyabroudi, Dr Bader Alqahtani, Dr Sahar Abdulaziz, Dr Kefah Alshebber, Dr Carrie Hoppes, David Fear, and Mohammed Almotairi. They also acknowledge the Clinical and Translational Science Institute (CTSI) for helping with participant recruitment.

Ethics Approval

This study was approved by the Institutional Review Board of the University of Pittsburgh, and all participants provided informed consent prior to participating in the study.

Funding

This study was supported in part by the National Institutes of Health through grants UL1TR001857 and 5-R21-DC-012410-02.

Disclosure

The authors completed the ICJME Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest. Author B.N. Klatt discloses an NIH Training in Auditory and Vestibular Neuroscience Grant (#T32-DC011499 – Trainee).