Validity of physical activity measures in individuals after total knee arthroplasty

Abstract

OBJECTIVE

To determine the concurrent criterion-related validity of two activity monitors in comparison to the criterion method of indirect calorimetry in older adults after total knee arthroplasty (TKA).

DESIGN

Validation study.

SETTING

Subjects completed 9 increasingly demanding daily activities in a research laboratory; each activity was performed for 7 minutes, for a total of 80-minutes while the activity monitors and criterion method were used concurrently.

PARTICIPANTS

Twenty-one subjects, 67% female, mean age 68±7 years old, and BMI 29±4.

INTERVENTIONS

not applicable.

MAIN OUTCOME MEASURE

Energy expenditure (EE) in kcal/min measured by accelerometer-based and multisensor-based monitors, and the criterion method. Validity was assessed by paired t-test, intra-class correlation coefficient (ICC), and Bland-Altman plots comparing the measurements from the activity monitors to the criterion method.

RESULTS

Measurements from the accelerometer-based monitor were significantly lower than those of the criterion method across all walking and non-walking activities. The underestimations ranged from 40% to 100%. The accelerometer-based monitor demonstrated small to moderate agreement compared to the criterion method (ICCs from 0 to 0.38). Measurements from the multisensor-based monitor were significantly lower than the criterion method during several non-walking activities; yet, the differences were minor (2% to 19%). Measurements from the multisensor-based monitor during walking activities were not different compared to the criterion method. The multisensor-based monitor demonstrated moderate to excellent agreement with the criterion method (ICCs from 0.48 to 0.81).

CONCLUSION

The multisensor-based monitor showed better criterion-related validity than the accelerometer-based monitor, and should be considered as a tool to measure physical activity in individuals after TKA.

Physical activity (PA) is an important construct to be assessed in older adults as it relates to future mobility disability, chronic disease, and mortality.¹ Older adults who underwent total knee arthroplasty (TKA) generally have an inactive lifestyle due to persistent knee pain and functional limitations caused by several decades living with knee osteoarthritis.^2,4 Thus, investigating the validity of measures to assess PA in individuals after TKA is warranted.

Commonly used activity monitors are accelerometer-based and multisensor-based.^5-13 A beneficial aspect of these devices is that they capture PA at several intensities, ranging from sedentary to moderate. The ability of these devices to assess PA at sedentary and light intensities is important since older adults who undergo TKA perform most of their daily activities in light intensities.^2,4 Therefore, research is needed in this population to validate and compare the performance of these devices during PA across a variety of intensities.

Numerous studies have assessed the validity of these activity monitors in separate investigations reporting mixed results.^5-12 However, validity of the devices cannot be compared as they have not been assessed concurrently in the same study and studies have used diverse population and methods to estimate PA. In older adults, we are aware of only one study that investigated the concurrent validity of the two activity monitors against indirect calorimetry. This study reported a moderate association between measures of energy expenditure from the accelerometer-based monitor and indirect calorimetry, and a strong association between the multisensor-based monitor and indirect calorimetry, during a protocol that included various activities.¹³ To our knowledge, no studies have concurrently compared the performance of these activity monitors against a criterion method during individual activities of several intensity levels in older adults with TKA.

Investigating the concurrent validity of activity monitors to measure individual daily-activities performed by individuals after TKA is relevant as their functional limitations and gait deviations may contribute to their inactive lifestyle, and may affect accelerometry data.^14,15 Comparing the performance of an accelerometer-based to a multisensor-based monitor will also provide evidence for informed decisions of clinicians and researchers when choosing a monitor to measure PA in people with similar characteristics. The purpose of this study was to determine the concurrent criterion-related validity of two activity monitors against indirect calorimetry in measuring energy expenditure during sedentary to moderate intensity activities in older adults after TKA, and we hypothesized similar validity of both activity monitors.

METHODS

This validation study was conducted at the Pittsburgh Claude D. Pepper Older Americans Independence Center, University of Pittsburgh, from August/2009 to March/2011. All subjects recruited signed the informed consent approved by the University’s Institutional Review Board. Invitation letters were sent to individuals who underwent TKA. Eligibility screening was done over the phone by KSB and in-person by GJA. Inclusion criteria were age 50 years or older and prior unilateral TKA due to end-stage knee osteoarthritis. Exclusion criteria were two or more falls within the previous year, inability to walk for 31 meters without assistive device, history of cardiovascular disease or uncontrolled high-blood pressure, severe visual impairment, lower-extremity amputation, and neurological disorder.

Measures

Subjects participated in one testing visit. A questionnaire was used to collect demographic information along with subjects’ height and weight. We characterized physical function using the Western Ontario and McMaster Universities (WOMAC) Osteoarthritis Index physical function subscale. The WOMAC physical function subscale has 17 items, each scored on a 5-point Likert scale with a total of 68 points. Higher scores represent worse physical function. The WOMAC is a reliable and valid measure of physical function in individuals with knee osteoarthritis.^16,17

Activity Monitors

The Actigraph (ACT^a), GT1M model, is a small uniaxial accelerometer (2 × 1.5 × 0.6 inches in size) worn at the waist level that measures body acceleration in activity counts. The ActiLife 4 software^a was used to initialize the monitor and download its data. The ACT^a was set to collect data at 1-minute epochs. The software converts activity counts per minute (counts/min) into energy expenditure (EE) in kilocalories per minute (kcal/min) using the work-energy theorem for counts/min less than 1952: (0.0000191 × counts/min) × Body mass in kg;¹⁸ and the Freedson’s equation for counts/min equal to or higher than 1952: (0.00094 × counts/min) + (0.1346 × Body mass in kg) - 7.37418.¹⁹ The ACT^a has demonstrated good to excellent test-retest reliability (>0.75) to measure PA in healthy young adults.²⁰ In our laboratory, the test-retest reliability of measures of EE assessed by the ACT^a in older adults was excellent during lie-down (ICC=0.84), and treadmill walking (ICC=0.94).

The Sensewear Pro3 Armband (SWA^b), is a small monitor (3.4 × 2.1 × 0.8 inches in size) worn on the right upper arm. It combines information from a biaxial accelerometer and physiological indicators, including heat flux, galvanic signal, and skin temperature. The multi-sensor information is integrated with subjects’ age, height, weight and gender, and was processed by the InnerView Professional Research software v6.1. The software calculates EE expressed in kcal/min using proprietary algorithms. The SWA^b has demonstrated excellent test-retest reliability (ICC=0.97) of measures of PA performed by healthy young adults.²¹ In our laboratory, the test-retest reliability of EE measured in older adults by the SWA^b ranged from good to excellent during lie-down (ICC=0.78), and treadmill walking (ICC=0.94).

Indirect Calorimetry

The MedGraphics VO2000^c was used as a criterion method to measure EE. This is a gas analyzer system that has been widely used in several populations.^22-25 The system includes a metabolic unit, a neoprene facemask that covers the subjects’ mouth and nose, and a breathing valve that connects to the mask. The device provides data on minute-by-minute oxygen consumption expressed in milliliter-per-minute that were converted into kcal/min using the formula: mL/min × 4.9/1000. The appropriate breathing valve was used for each activity, according to manufacturer’s recommendation. The VO2000^c has shown good validity to distinguish EE across various activities intensities.^26,27

Testing Protocol

Subjects were instructed not to eat for 2 hours and not to exercise 3 hours prior to the testing visit. All data were collected by the same tester (GJA). The VO2000 was calibrated 30 minutes prior to each testing visit following the manufacturer’s instructions. The same computer was used to initialize and download data from the two activity monitors and VO2000 to ensure time synchronization. Information on subjects’ gender, height, weight, date of birth, and limb dominance was entered into the software as required for each device. The ACT^a was placed on the subjects’ waist, aligned with the right anterior iliac crest using a belt clip, and the SWA^b was placed over the triceps of subjects’ right arm at midpoint between elbow and shoulder. The ACT^a was initialized using the computer while the SWA^b turned on as soon as it acclimatized to the subjects’ skin. Finally, the VO2000^c’s facemask was fitted over the subjects’ nose and mouth, and adjusted for comfort.

Data was concurrently collected using the ACT^a, SWA^b and VO2000^c while subjects performed 7-minutes bouts of 9 activities that mimicked habitual daily-living (Text Box 1). During treadmill-walk, subjects were encouraged to hold onto the handrails for safety. Throughout all activities except stand-and-talk, subjects were instructed not to talk. The entire testing protocol lasted 80-minutes. For each activity, the 7-minutes bout began after the subjects reached steady-state, which took one-to-three minutes. The steady-state period was defined as reaching a plateau of oxygen consumption in the VO2000, with values fluctuating ≤1 mL/kg/min for non-walking activities, and ≤3 mL/kg/min for walking activities. Rest periods of at least two-minutes were given between activities. At the end of testing, data from the ACT^a and SWA^b were downloaded into each device’s software. Data on kcal/min was matched minute-by-minute for the three devices. The first and last minutes of data collection for each activity were not used in the analysis as they did not represent the whole minute in steady-state. Measures of kcal/min from the three devices were calculated by averaging the remaining 5-minutes of each activity.

Sample Size and Data Analysis

Sample size was based on paired comparisons of measures from each activity monitor against the criterion method. We pre-determined that a sample size of 20 subjects would provide 80% power to detect a 15% difference in EE, 2-tailed test, alpha 0.025 to account for two comparisons. We assumed a correlation of 0.50, and used data from EE measured by the VO2000^c during self-selected walking from our pilot reliability study for this estimation (mean EE of 4.03±0.86 kcal/min).

Descriptive statistics of continuous variables included mean and standard deviation or median and 25-75 percentiles according to data distribution assessed by Shapiro-Wilk test. Categorical variables were described in counts and frequencies. Raw differences and percentage differences were calculated for measures of ACT^a and SWA^b in comparison to VO2000^c for each activity. Validity was assessed by paired t-test and intra-class correlation coefficient (ICC) comparing the measurements from each activity monitor to the measurements from the VO2000^c. ICC used absolute agreement definition and 2-way random effects model (ICC_2,k). To identify if differences between measurements from the VO2000^c and the portable monitors were above measurement error, we compared the differences to the standard error of measurement (SEM) of the VO2000^c (0.26 kcal/min), which was calculated from repeated-measures of self-selected walking in older adults (reliability=0.92, standard deviation=0.91 kcal/min) using the following formula: SEM= SD √1-reliability. We also created Bland-Altman plots to assess systematic bias. These procedures were done for each of the 9 activities and for EE of the entire testing protocol (80-minute). SPSS statistical software 17.0.2^e and XLSTAT^f were used for hypothesis testing and Bland-Altman plots respectively.

RESULTS

Invitation letters were mailed to 42 subjects whereas 37 called to request study information. From these, 4 were not eligible due to bilateral TKA (2), falls in past year (1) and use of a cane for walking (1). The 33 eligible subjects were scheduled according to their availability until the target sample size was reached. Characteristics of the 21 subjects who participated indicated that our sample is a good representation of individuals following TKA (Table 1).

TABLE 1.

Demographic and biomedical characteristics of study subjects. Numbers represent mean ± SD, unless otherwise indicated.

Variables	N = 21
Age	68.2 ± 7.0
Sex – female (%)	14 (67)
Height (cm)	167.8 ± 7.9
Weight (kg)	86.3 ± 10.9
BMI	0.29 ± 0.04
Married – n (%)	12 (57)
Last grade in school – n (%)
High school	11 (52)
College	10 (48)
Years after TKA	3.3 ± 0.3
WOMAC-Physical Function	14.6 ± 9.6

BMI: body mass index; TKA: total knee arthroplasty; WOMAC: Western Ontario and McMaster Universities Osteoarthritis Index.

During non-walking activities the ACT^a significantly underestimated measures from the VO2000 (Figure 1). The differences ranged from 1.13 to 2.3 kcal/min (93 to 100%) and were above measurement error (SEM). The ICCs ranged from 0.00 to 0.01, demonstrating no agreement. The Bland-Altman plots demonstrated systematic bias, where the differences between measures from the ACT^a and VO2000^c became larger as values of EE increased (Figure 2). The SWA^b significantly underestimated EE during sit-and-read, computer-work, and stand-and-load shelves, whereas it significantly overestimated EE during mopping as compared to the VO2000^c. However, these differences were small, ranging from 0.02 to 0.48 kcal/min (2 to 19%), and did not exceeded the SEM, except for mopping (Figure 1). The ICCs ranged from 0.60 to 0.81, demonstrating moderate to good agreement. Bland-Altman plots indicated no systematic bias (Figure 2).

FIGURE 1.

Measures of energy expenditure in kcal/min estimated by the Actigraph (ACT)^a, and the SenseWear Armband (SWA)^b, in comparison to the VO2000^c (criterion method) during non-walking activities. Numbers represent mean ± SD, unless otherwise indicated.

Diff: = difference between VO2000 and portable monitor; ICC: intra-class correlation coefficient; §: not able to calculate due to lack of variance (all ACT values were zero); ¶: Significant difference between measurements from VO2000 and ACT (p<0.025); †: Significant difference between measurements from VO2000 and SWA (p≤0.025).

FIGURE 2.

Bland-Altman plots illustrating the differences between energy expenditure, expressed in kcal/min, estimated by the Actigraph (ACT)^a, SenseWear Armband (SWA)^b, and the criterion method (VO2000)^c, across the 6 non-walking activities.

Blue solid line indicates difference between measures from portable device and VO2000. Blue dashed line indicates 95% confidence interval around the difference between devices. Red dashed line indicates the 95% limit of agreement between pooled mean from both devices.

During walking activities the ACT^a significantly underestimated measures from the VO2000^c, with differences ranging from 1.9 to 3.0 kcal/min (40% to 79%), all above SEM (Figure 3). The ICCs ranged from small to moderate (0.12 to 0.38). The differences between measures from the SWA^b and VO2000^c during walking were small and not significant. The differences were below the SEM and ranged from 0.06 to 0.44 (2% to 10%). The ICCs were moderate and ranged from 0.48 to 0.63. The Bland-Altman plots for both ACT and SWA against the VO2000 indicated no systematic bias (Figure 4).

FIGURE 3.

Measures of energy expenditure in kcal/min estimated by the Actigraph (ACT)^a and the SenseWear Armband (SWA)^b in comparison to the VO2000^c (criterion method) during walking activities. Numbers represent mean ± SD, unless otherwise indicated.

Diff: difference between VO2000 and activity monitor; ICC: intra-class correlation coefficient; ¶: Significant difference between measures from VO2000 and ACT (p<0.025); SS: self-selected.

FIGURE 4.

Bland-Altman plots illustrating the differences between energy expenditure, expressed in kcal/min, estimated by the Actigraph (ACT)^a, SenseWear Armband (SWA)^b, and the criterion method (VO2000)^c, across the 3 walking activities.

Blue solid line indicates difference between measures from portable device and VO2000. Blue dashed lines indicate 95% confidence interval around the difference between devices. Red dashed lines indicate the 95% limit of agreement between pooled mean from both devices.

Results from the EE to complete the 80-minutes testing protocol indicated that the ACT^a significantly underestimated the VO2000c by 78% (42.8±35.0 versus 189.8±38.8 kcal, p=0.001) with small agreement (ICC=0.11). The difference between SWA^b and VO2000^c was 5% (180.3±49.8 versus 189.8±38.8 kcal, p=0.26) with good agreement (ICC=0.78; F_1,20=1.35, p=0.260). The Bland-Altman plots indicated no systematic bias of both activity monitors against the VO2000^c (plots not shown).

DISCUSSION

To our knowledge, this is the first study reporting and comparing the validity of measures of EE concurrently estimated by the ACT^a, SWA^b, and VO2000^c during short-bouts of typical daily-living activities, ranging from sedentary to moderate intensities, in older adults after TKA. Our results provided evidence that the validity of the SWA^b is superior to that of the ACT^a in measuring EE during non-walking and walking activities, and during the entire protocol. These findings are informative to the research and clinical community as they provide support for a widely used monitor, the SWA, to assess daily PA in older adults after TKA.

For the non-walking activities our results agree with studies that reported important underestimations of EE by the ACT^a, especially when estimating energy spent during activities involving upper-body movement in samples of young and middle-aged adults.^7,29,30 The validity of the SWA^b during non-walking activities has also been questioned in studies in population of healthy-young and obese adults, adults with cancer, and older adults with chronic obstructive pulmonary disease.^31-34 In our study, EE during mopping was overestimated by the SWA^b. As mopping was performed in a limited area, the subjects did not walk around to perform the task, but rather mainly used their upper-body, which possibly explain why the SWA^b overestimated EE since this device is worn on the arm. Other studies have also reported overestimations of EE by the SWA^b during activities that require upper-body movement (e.g., computer work, fold laundry, and sweeping).^9-11 While one study reported overestimations similar to ours (15%),⁹ two recent studies in hip osteoarthritis in rheumatoid arthritis reported much higher overestimation of EE by the SWA^b (up to 119%) during sweeping.^10,11 However, these studies did not describe how the task was performed. We have observed that individuals with arthritis of the lower extremities tend to compensate with upper-body to perform the task, resulting in higher readings by the SWA^b. Overall, the capability of the SWA^b in estimating EE during non-walking activities seems to be better than the ACT^a. Since the ACT^a is worn on the waist it may not be suitable to measure activities that do not involve ambulation, a limitation that has been previously discussed in several studies.^5,8,13,29

For walking activities, our findings also showed better validity for the SWA^b in comparison to the ACT^a. The validity of the SWA^b has been supported by studies in healthy-young adults,³⁰ and middle-aged adults with heart disease.³¹ Yet, several studies reported EE overestimations by the SWA^b (31% to 93%).^{9-12,32,34-37} The differences across results may be due to divergent methodologies and population used by the studies. For example, some studies did not require subjects to be in steady-state during experimental conditions,^9,10 whereas steady-state is important to establish a fair comparison across devices.³⁷ Our subjects took about one-to-three minutes to reach steady-state in each activity. If measures from the SWA^b were taken while measures from indirect calorimetry are still going up, the SWA^b readings would be higher.³⁷ Additionally, some studies have used older SWA^b software (e.g., v3.0 and v4.0), when the software algorithms were still being refined to account for different body sizes and age.^29,36 Another study that reported large estimation of the SWA over the VO2000 was in subjects with hip osteoarthritis scheduled for total hip arthroplasty.¹¹ It is possible that hip pain and associated gait deviations (i.e., limping) could have affected body-acceleration. In our study the subjects did not limp during walking. In other studies they included subjects with pulmonary diseases without accounting for their difficulty breathing, which may also have affected the readings from indirect calorimetry.^34,35

Results from several prior studies have also questioned the validity of the ACT during walking and agree with our results.^5,8,13,29 In contrast to our results, a number of studies reported good validity of the ACT^a (differences with indirect calorimetry ranged from 4% to 12%) during walking in young to middle-aged normal-weight adults.^38-41 The inaccuracies of the ACT^a in our study could likely be due to excessive abdominal fat observed in a large number of our subjects. As the ACT^a is worn around the waist, the abdominal fat may have kept the device in non-vertical position (despite our effort to re-adjust it to vertical), restricting the readings of vertical acceleration during walking. However, this hypothesis could not be tested, as we did not measure abdominal fat.

Our results also disagree with one study in healthy-young adults that tested the ACT^a and SWA^b concurrently against indirect calorimetry during treadmill walk.²⁹ Values of the ACT^a during self-selected and fast-walk were accurate, while the SWA^b overestimated measurements of EE in slow, self-selected, and fast-walk (25% to 59%). The discrepancies observed between results for the ACT^a might be due to their sample of lean adults. As discussed above, it seems that the ACT^a shows better validity in lean individuals than in overweight-obese ones. In regards to the SWA^b, they used an older SWA^b software (v3.0) that was probably not as accurate as the one used in our study (v6.1). It is also possible that difference between protocols regarding activity intensity (i.e., usual walking speed in healthy-adults is higher than in older adults after TKA), and how task is performed (i.e., to use versus not to use handrail) could account for some of the differences.

The validity of the activity monitors was also tested during the entire testing protocol. The SWA^b showed good validity to measure EE for the entire 80 minutes of the protocol, while the ACT^a has shown poor validity. We are aware of two studies that investigated the concurrent validity of the ACT^a and SWA^b against indirect calorimetry to measure EE to complete the entire testing protocol that included activities ranging from sedentary-to-moderate intensities.^11,42 One study was in older adults with chronic obstructive pulmonary disease during a 59-minutes protocol,¹¹ and the other was in healthy young to middle-aged adults in a 120-minutes protocol.⁴² Their findings agree with ours on the better validity of the SWA^b in comparison to the ACT^a. This finding is especially important as PA is typically assessed in free-living continuously for several days, several hours a day.

Study limitations

Although we have chosen to compare the validity of the ACT^a and SWA^b, we acknowledge that many activity monitors are available on the market but were not tested. These devices were chosen since they are commonly used in arthritis-related research and they are capable to estimate PA at low intensities. We are also aware of many ACT^a equations used to estimate EE, but our intention was to test these devices to mimic clinical practice by using the default options. Thus, devices and software were used as recommended by the manufacturers. Moreover, we have used the monitors and software available at the time this study was planned. Currently the ACT uses a triaxial rather uniaxial accelerometer and its validity may be better.^43,44 Although our protocol included a variety of gradually demanding activities typically performed by older adults, we were limited by using an indirect calorimetry unit not portable. This limitation prevented us from including activities that require moving around such as walking over ground or stair climbing, which should be considered when interpreting the results. Last, although our results cannot be generalized to the overall population of individuals with arthritis of the lower extremities, we believe that the results can be generalized to the population of individuals with knee osteoarthritis who undergo TKA since our sample was a good representation of this population.

CONCLUSIONS

The SWA^b demonstrated better criterion-related validity than the ACT^a in measuring EE across all activities tested in the study protocol performed by older adults after TKA. Since this population tends to be overweight, sedentary, and perform most daily activities and PA at light intensity, the SWA^b seems to be a better choice of activity monitor to assess PA behavior.

Text Box 1. Sequence of activities performed by each subject during testing protocol.

1. Lie-down: subjects laid on a treatment table and remained as immobile as possible;

2. Sit-and-read: subjects sat quietly on a chair and read an 8^th grade level story;

3. Computer-work: subjects sat at a computer and typed, at their own pace, the text from an 8^th grade level story

4. Stand-and-talk: subjects were standing in front of the tester, and were asked a few standardized questions about their commute and daily activities;

5. Stand-and-load shelves: subjects moved 0.45 Kg cans from a counter (waist level) to a shelf (eye level) alternating arms at a pre-set pace (40 times/min). A metronome (Seiko DM70)^d was used to set the pace of this activity;

6. Mopping: subjects mopped the floor covering an area of 3m²;

7. Walk on a treadmill at self-selected speed;

8. Walk on a treadmill at slow speed: treadmill self-selected walk speed was reduced by 0.18 m/sec (0.40 mph). A decrease in gait speed of 0.18 m/sec is considered clinically meaningful ²⁸;

9. Walk on a treadmill at fast speed: treadmill self-selected walk speed was increased by 0.18 m/sec.

ABBREVIATIONS

ACT

Actigraph

EE

Energy expenditure

PA

Physical activity

SEM

Standard Error of Measurement

SWA

SenseWear Armband

TKA

Total knee arthroplasty

WOMAC

Western Ontario and McMaster Universities Osteoarthritis Index