Abstract
Objective
This study explored the criterion-related validity and test-retest reliability of the modified RESIDential Enviroment (RESIDE) physical activity questionnaire and whether the instrument’s validity varied by body mass index (BMI), education, race/ethnicity, or employment status.
Design
Validation study using baseline data collected for randomized trial of a weight loss intervention.
Method
Participants recruited from health departments wore an ActiGraph accelerometer and self-reported non-occupational walking, moderate and vigorous physical activity on the modified RESIDE questionnaire. We assessed validity (n=152) using Spearman correlation coefficients (SCC), and reliability (n=57) using intraclass correlation coefficients (ICC).
Results
When compared to steps, moderate physical activity, and bouts of moderate/vigorous physical activity measured by accelerometer, these questionnaire measures showed fair evidence for validity: recreational walking (SCC 0.23–0.36), total walking (SCC 0.24–0.37), and total moderate physical activity (SCC 0.18–0.36). Correlations for self-reported walking and moderate physical activity were higher among unemployed participants and women with lower BMIs. Generally no other variability in the validity of the instrument was found. Evidence for reliability of RESIDE measures of recreational walking, total walking, and total moderate physical activity was substantial (ICC 0.56–0.68).
Conclusions
Evidence for questionnaire validity and reliability varied by activity domain and was strongest for walking measures. The questionnaire may capture physical activity less accurately among women with higher BMIs and employed participants. Capturing occupational activity, specifically walking at work, may improve questionnaire validity.
Keywords: validity, reliability, physical activity assessment, obesity, accelerometer
Introduction
Globally, the health impact of physical inactivity is similar to that of smoking.1 Trends suggest that physical inactivity2 and overweight and obesity3 are more prevalent among low-income women compared to men and high-income groups. Increasing physical activity (PA) is an important component of weight control and health improvement. However, valid, reliable measures are needed to monitor and evaluate PA in high-risk populations.4,5
Self-report questionnaires are feasible,6 cost-effective4 PA measures, but are subject to recall and response biases.2,4 For example, while 51% of Americans self-reported 30 or more minutes of PA at least five days per week in 2003–2004, fewer than 5% achieved this level of PA according to accelerometer measures.2 Further, factors such as sex, body mass index (BMI),5 education, and race/ethnicity may influence participant understanding and interpretation of PA questionnaires.7–9
The RESIDential Environment (RESIDE) questionnaire was developed to assess PA, specifically transportation and recreational walking, among Australian women. This instrument has been shown to rapidly and reliably6,10,11 assess walking among Australian adults,6 midlife Canadian women,11 and older Chinese adults.10 A modified version of this questionnaire was used in the Weight-Wise II Study, a group-based weight loss intervention for low-income women with a walking focused PA component. However, the reliability and validity of the RESIDE questionnaire have not been assessed in low-income overweight or obese populations. Variations in validity by participant BMI, education, race/ethnicity and employment status also have not been assessed.10 Given reporting differences in other behaviors,12,13 overweight and obese individuals may respond differently to the RESIDE questionnaire compared to normal weight respondents. The objectives of this study were to assess the criterion-related validity and test-retest reliability of the modified RESIDE questionnaire and to determine whether the validity of this instrument varied by BMI, education, race/ethnicity, or employment status among overweight and obese women.
Methods
This validation study was conducted using baseline assessments of participants in the Weight-Wise II Study (n=189), a randomized controlled trial evaluating a group-based weight loss intervention program for overweight and obese, low-income, midlife women, described in detail elsewhere.14,15 The Weight-Wise II Study enrolled participants at six North Carolina county health departments located in two higher density (population >130,001), two intermediate density (population 46,501 to 130,000), and two lower density counties (population < 46,500).16 All assessments included in these analyses were conducted prior to the intervention. All participants provided written informed consent and the study was approved and monitored by the Institutional Review Board at the University of North Carolina – Chapel Hill.
Each of six sites was asked to enroll 40 women during an eight-week period. Inclusion criteria were: 1) aged 40–64 years; 2) BMI 27.5–45 kg/m2; 3) willing to lose 5% or more of initial body weight and follow recommendations for healthy dietary and PA patterns; 4) English speaking; 5) household income ≤ 250% of United States (US) federal poverty guidelines; and 6) able and willing to give informed consent. Participants received gift cards for attending the enrollment visit ($10), for wearing the accelerometer ($20), for completing the initial telephone-administered surveys ($10), including the modified RESIDE PA questionnaire, and for completing the test-retest reliability phone call ($20).
At the enrollment visit, participants self-reported sociodemographic information, prior weight loss and coronary hear disease risk factors. Weight, height, and blood pressure were objectively measured using protocols described elsewhere14 and BMI was calculated (kg/m2) to confirm study eligibility. Participants were fitted with an ActiGraph GT1M accelerometer (ActiGraph, Pensacola, FL) to directly measure PA. The ActiGraph can accurately measure steps and counts during ambulatory activity17 and sedentary behavior18 in adults. Participants were instructed to wear the accelerometers attached to an elastic belt over the right hip at waist level during all waking hours for the next 7 days, except while showering or swimming. Accelerometers were configured to collect counts and steps in the uniaxial mode using a 10-second epoch. Accelerometers were returned in a postage-paid priority mailer to the research office where data were downloaded and stored.
Accelerometer wear time (min/d) was computed by subtracting non-wear time from 24h. Non-wear time was defined as a period of 60 minutes or more of zero counts, allowing 2 minutes of counts > 0 but ≤ 100. A minimum of 10 hours of wear time on at least 4 days was required for inclusion in these analyses.
Accelerometer measured PA was calculated as average counts/min, and then converted to min/d of activity by intensity level. A number of studies of adults provide count thresholds or cutpoints to distinguish moderate to vigorous-PA from other forms of less intense activity. We used the Troiano et al.2,19 definition from the US National Health and Nutrition Examination Survey. Vigorous intensity was defined as ≥ 5999 counts/min, moderate intensity as 2020–5998 counts/min, and light intensity as 101–2019 counts/min. Bouts of moderate to vigorous-PA were defined as intervals ≥ 10 minutes with accelerometer counts ≥ 2020 counts/min, allowing up to 2 minutes of interruption with counts below the threshold.
The modified RESIDE PA questionnaire was administered to participants by phone 1–3 weeks (median 20 days) post-enrollment. The original RESIDE Neighborhood PA Questionnaire is a 21-item self-administered instrument in which participants recall the frequency, duration, and destination of their walking (for transport and recreation) within and outside of their neighborhood, and cycling activity in a usual week.6 We modified the questionnaire by combining the walking items into two location-neutral walking measures: walking for transportation and walking for recreation. We also expanded the cycling item to include all non-walking moderate- and vigorous-PA, revised PA locations for regional appropriateness (e.g., fitness center replaced beach), and adapted the questionnaire for telephone administration. The 16-item modified questionnaire addressed four activity domains: walking for transportation; walking for recreation, health or fitness (“walking for recreation”); moderate-PA; and vigorous-PA. Participants were asked whether they engaged in each activity domain and if so, the location, frequency, and usual weekly duration of that activity (supplementary material). We divided weekly durations by seven to calculate a daily average duration (min/d) for each domain. Three summary measures were computed: total walking (sum of walking for recreation and walking for transport), total moderate-PA (sum of total walking and non-walking moderate-PA), and total activity (sum of total walking, moderate- and vigorous-PA).
Criterion-related validity is the ability of an instrument to accurately measure the construct it is intended to measure.20 We explored the ability of the modified RESIDE questionnaire to accurately capture the frequency, duration, and intensity of participant PA by comparing PA measured by accelerometer to self-reported PA using Spearman correlation coefficients (SCC). SCC were calculated for three accelerometer measures: steps/d, moderate-PA (min/d), and bouts of moderate to vigorous-PA (min/d) when compared to three questionnaire measures: walking for recreation (min/d), total walking (min/d), and total moderate-PA (min/d). SCC were calculated for the entire sample (n=152) and stratified by BMI (27.5 – < 37, ≥ 37), education level (< 13 years, ≥ 13 years), race/ethnicity (non-Hispanic black, non-Hispanic white), and employment status (employed, unemployed) to explore differences in validity between subgroups.
Test-retest reliability gauges the consistency of repeated measurements, with high reliability indicating that variation is attributable to true difference in the population rather than measurement error.20 To assess test-retest reliability of the modified RESIDE questionnaire, we re-administered the questionnaire by telephone to a consecutive sub-sample (n=57) a median of 23 days after initial administration. The Shrout and Fleiss method based on a one-way analysis of variance was used to calculate intraclass correlations (ICC) between repeated measures.21 The ICC corresponds to the variance between participants expressed as a proportion of total variance (measurement error plus between subject variance).
Validity and reliability were classified into five categories: poor (0 to < 0.2), fair (0.2 to < 0.4), moderate (0.4 to < 0.6), substantial (0.6 to < 0.8), and almost perfect (0.8–1.0).22 Stratified PA estimates and correlations were considered substantially different if the index group mean was not within the 95% confidence interval for the comparison group mean. No adjustments were made for multiple comparisons. All analyses were conducted using SAS version 9.3 (Cary, NC).
Results
Of 189 women enrolled in the Weight-Wise II Program, 152 (80%) were included in these analyses. Thirty-seven women were excluded: 34 did not meet the minimum required accelerometer wear time, one did not wear the accelerometer, and two did not complete the modified RESIDE questionnaire.
The mean age of participants was 51.1 years, 53% were non-Hispanic black, and mean educational achievement was 13 years (Table 1). More than half of participants were employed (54%) and lacked health insurance (57%), and 17% reported an annual household income of less than $10,000. Participants’ mean BMI was 37.3 kg/m2.
Table 1.
Sociodemographic and coronary heart disease (CHD) risk factor characteristics of Weight-Wise II Study participants
| Sociodemographics: | Mean ± SEM or % (n) (n=152) |
|---|---|
| Age (y) | 51.1 ± 0.6 |
| Race/Ethnicity | |
| Non-Hispanic black | 53 (81) |
| Other a | 47 (71) |
| Educational achievement (y) | 13.0 ± 0.2 |
| Living with spouse or someone like a spouse | 47 (71) |
| Adults (≥ 18 y) in household | 1.9 ± 0.1 |
| Children (< 18 y) in household | 0.8 ± 0.1 |
| Currently employed | 54 (82) |
| Lack health insurance | 57 (86) |
| Annual household income < $10,000 | 17 (25) |
|
CHD Risk Factors:
| |
| Weight (kg) | 99.7 ± 1.1 |
| Body mass index (kg/m2) | 37.3 ± 0.4 |
| Overweight (25 ≤ BMI < 30) | 5 (8) |
| Obese, class I (30 ≤ BMI < 35) | 24 (37) |
| Obese, class II (35 ≤ BMI < 40) | 39 (60) |
| Obese, class III (BMI ≥ 40) | 31 (47) |
| Systolic blood pressure (mm Hg) | 125.5 ± 1.5 |
| Diastolic blood pressure (mm Hg) | 82.8 ± 1.0 |
| Current cigarette smoker | 17 (25) |
| Diagnosed with high blood pressure | 49 (75) |
| Treated with blood pressure medication | 46 (69) |
| Diagnosed with high blood cholesterol | 40 (60) |
| Treated with cholesterol medication | 30 (45) |
| Diagnosed with diabetes | 20 (31) |
| Positive family history for CHD | 28 (43) |
| Known CHDb | 15 (22) |
66 Non-Hispanic White, 2 Hispanic White, 2 non-Hispanic American Indian/Alaska Native, and 1 non-Hispanic unknown
Known CHD based on an affirmative response to any of the following: 1. Has the doctor ever told you that you have had a heart attack?; 2. Have you had by-pass surgery or another procedure to open blood vessels in your heart like angioplasty, stent placement or atherectomy?; 3. Has a doctor ever told you that you have angina; that is chest pain that comes from your heart?
BMI: body mass index; CHD: coronary heart disease; SEM: standard error of the mean; y: year
Participants reported a median of 0 min/d (walking transportation, vigorous-PA) to 9 min/d (total moderate-PA and total PA) of activity in each domain on the modified RESIDE questionnaire (Table 2). Unemployed participants reported more total walking (median 9 vs. 4 min/d), moderate-PA (median 12 vs. 9 min/d), and total activity (median 12 vs. 9 min/d). There were no meaningful differences in self-reported PA by BMI, education, or race/ethnicity.
Table 2.
Physical Activity (PA) measured by modified RESIDEa questionnaire and accelerometer for the Weight-Wise II Study sample and stratified by body mass index (BMI), education, race/ethnicity, and employment status.
| All | BMI | Education | Race/Ethnicity | Employment Status | |||||
|---|---|---|---|---|---|---|---|---|---|
|
| |||||||||
| < 37 | ≥ 37 | ≤ 13 yr | > 13 yr | Black | White | Not Employed | Employed | ||
| (n=152) | (n=75) | (n=77) | (n=95) | (n=57) | (n=81) | (n=68) | (n=70) | (n=82) | |
| Mean ± SEM Median (IQR) |
Mean ± SEM Median (IQR) |
Mean ± SEM Median (IQR) |
Mean ± SEM Median (IQR) |
Mean ± SEM Median (IQR) |
|||||
| Modified RESIDE Questionnaire | |||||||||
|
| |||||||||
| Walking for Transport (min/d) | 0.6 ± 0.2 | 0.7 ± 0.3 | 0.6 ± 0.3 | 0.9 ± 0.3 | 0.2 ± 0.2 | 1.0 ± 0.4 | 0.2 ± 0.1 | 0.7 ± 0.4 | 0.6 ± 0.3 |
| 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | |
|
| |||||||||
| Walking for Recreation (min/d) | 11 ± 1.6 | 12 ± 1.9 | 11 ± 2.5 | 10 ± 2.3 | 13 ± 1.9 | 11 ± 1.7 | 12 ± 2.9 | 13 ± 2.8 | 9.6 ± 1.6 |
| 4 (0–17) | 6 (0–19) | 1 (0–13) | 0 (0–13) | 9 (0–18) | 6 (0–13) | 0 (0–17) | 5 (0–19) | 4 (0–13) | |
|
| |||||||||
| Total walking (min/d) | 12 ± 1.6 | 13 ± 1.9 | 11 ± 2.5 | 11 ± 2.3 | 13 ± 1.9 | 12 ± 1.7 | 12 ± 2.9 | 14 ± 2.8 | 10 ± 1.6 |
| 4 (0–17) | 9 (0–19) | 3 (0–13) | 2 (0–17) | 9 (0–18) | 9 (0–17) | 1 (0–17) | 9 (0–20) | 4 (0–14) | |
|
| |||||||||
| Total moderate-PAb (min/d) | 19 ± 2.2 | 20 ± 2.6 | 19 ± 3.6 | 16 ± 2.7 | 25 ± 3.9 | 19 ± 2.8 | 19 ± 3.7 | 22 ± 3.6 | 17 ± 2.8 |
| 9 (0–27) | 11 (0–30) | 9 (0–24) | 6 (0–25) | 17 (4–32) | 9 (0–28) | 9 (0–25) | 12 (0–30) | 9 (0–26) | |
|
| |||||||||
| Vigorous-PA (min/d) | 0.2 ± 0.1 | 0.1 ± 0.1 | 0.2 ± 0.2 | 0.2 ± 0.2 | 0.2 ± 0.2 | 02 ± 0.2 | 0.1 ± 0.1 | 0.4 ± 0.3 | 0.0 ± 0.0 |
| 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | |
|
| |||||||||
| Total activity (min/d) | 20 ± 2.2 | 20 ± 2.6 | 20 ± 3.6 | 16 ± 2.7 | 25 ± 3.9 | 20 ± 2.8 | 20 ± 3.7 | 22 ± 3.6 | 17 ± 2.8 |
| 9 (0–28) | 11 (0–30) | 9 (0–26) | 6 (0–26) | 17 (4–32) | 9 (0–29) | 9 (0–25) | 12 (0–33) | 9 (0–26) | |
|
| |||||||||
| Accelerometer | |||||||||
|
| |||||||||
| Wear time (min/d) | 803 ± 7 | 807 ± 10 | 799 ± 10 | 799 ± 9 | 810 ± 10 | 818 ± 10 | 786 ± 9 | 788 ± 10 | 816 ± 9 |
| 793 (748–845) | 797 (750–851) | 789 (743–840) | 789 (732–840) | 800 (757–851) | 814 (761–852) | 780 (738–814) | 781 (721–827) | 802 (766–852) | |
|
| |||||||||
| Counts/min | 148 ± 5 | 152 ± 7 | 144 ± 6 | 149 ± 6 | 147 ± 8 | 151 ± 7 | 145 ± 7 | 140 ± 7 | 155 ± 6 |
| 132 (109–192) | 134 (106–205) | 132 (111–168) | 131 (112–200) | 134 (99–190) | 132 (110–197) | 130 (103–171) | 125 (102–169) | 139 (115–197) | |
|
| |||||||||
| Steps (thousands/d) | 5.30 ± 0.16 | 5.81 ± 0.25 | 4.80 ± 0.19 | 5.29 ± 0.20 | 5.31 ± 0.27 | 5.31 ± 0.23 | 5.25 ± 0.24 | 4.88 ± 0.24 | 5.65 ± 0.21 |
| 4.87 (3.86–6.70) | 5.30 (4.14–7.34) | 4.51 (3.62–5.66) | 4.87 (3.84–6.78) | 4.87 (3.96–6.57) | 4.93 (3.87–6.17) | 4.67 (3.79–6.82) | 4.43 (3.48–6.54) | 5.24 (4.33–6.85) | |
|
| |||||||||
| Light-PA (min/d) | 311 ± 6.5 | 315 ± 9.6 | 308 ± 9.0 | 317 ± 8.1 | 303 ± 11.1 | 319 ± 8.9 | 301 ± 9.9 | 291 ± 9.2 | 328 ± 8.8 |
| 302 (257–364) | 294 (250–380) | 308 (258–358) | 308 (262–361) | 295 (232–368) | 304 (270–361) | 296 (236–360) | 288 (246–341) | 316 (282–380) | |
|
| |||||||||
| Moderate-PA (min/d) | 11 ± 0.8 | 12 ± 1.3 | 10 ± 1.1 | 11 ± 1.1 | 12 ± 1.3 | 12 ± 1.2 | 11 ± 1.2 | 11 ± 1.2 | 11 ± 1.2 |
| 9 (4–15) | 10 (4–16) | 7 (4–14) | 9 (4–13) | 10 (4–19) | 10 (5–14) | 7 (3–17) | 8 (3–15) | 9 (4–15) | |
|
| |||||||||
| Vigorous-PA (min/d) | 0.2 ± 0.1 | 0.1 ± 0.1 | 0.2 ± 0.2 | 0.2 ± 0.2 | 0.2 ± 0.2 | 0.2 ± 0.2 | 0.1 ± 0.1 | 0.3 ± 0.2 | 0.0 ± 0.0 |
| 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | 0 (0-0) | |
|
| |||||||||
| Bout minutesc (min/d) | 4 ± 0.6 | 4 ± 1.1 | 3 ± 0.7 | 3 ± 0.9 | 4 ± 0.9 | 4 ± 1.0 | 3 ± 0.8 | 4 ± 0.9 | 3 ± 0.9 |
| 0 (0–3) | 1 (0–5) | 0 (0–3) | 0 (0–3) | 0 (0–7) | 0 (0–3) | 0 (0–3) | 1 (0–4) | 0 (0–3) | |
Assessed by modified RESIDE’s Neighborhood Physical Activity Questionnaire.
Total reported moderate-PA (including walking)
PA ≥ moderate intensity that in bouts of ≥ 10 min duration allowing 2 min of interruption
BMI: body mass index; IQR: inter-quartile range; min/d: minutes per day; PA physical activity; SEM: standard error of the mean
Median accelerometer wear time was 793 min/d (13.2 h/d) with a median of 132 counts/min and 4871 steps/d (Table 2). By intensity, median activity levels were 302 min/d (5 h/d) for light, 9 min/d for moderate, and 0 min/d for vigorous-PA and moderate to vigorous-PA in bouts of 10 or more minutes. Women with lower BMIs completed more steps/d (median 5303 vs. 4508). Unemployed women completed fewer steps/d (median 4426 vs. 5242) and less light-PA (median 288 vs. 316 min/d). There were no differences in accelerometer-recorded PA by education or race/ethnicity.
Correlations between accelerometer and self-reported PA were poor to fair (SCC 0.18 to 0.37) (Table 3). Correlations were strongest for accelerometer-measured bouts of moderate to vigorous-PA (min/d) compared with self-reported walking for recreation (min/d), walking total (min/d), and total moderate-PA (min/d) (0.36, 0.37 and 0.36, respectively). Correlations were weakest for accelerometer-measured steps/d and moderate-PA (min/d) compared with self-reported total moderate-PA (min/d) (0.18 and 0.19, respectively).
Table 3.
Spearman correlation coefficient (SCC) of accelerometer and modified RESIDEa physical activity (PA) questionnaire measures for Weight-Wise II Study sample and stratified by body mass index (BMI) and employment status.
| All | BMI | Employment Status | |||
|---|---|---|---|---|---|
|
| |||||
| Accelerometer | 27.5 – < 37 | ≥ 37 | Not Employed | Employed | |
|
| |||||
| (n=152) | (n=75) | (n=77) | (n=70) | (n=82) | |
| Modified RESIDE | SCC (95% CI) | SCC (95% CI) | SCC (95% CI) | SCC (95% CI) | SCC (95% CI) |
| Stepsb | |||||
|
| |||||
| Recreational walkingc | 0.30 (0.15, 0.44) | 0.35 (0.13, 0.53) | 0.22 (0.00, 0.42) | 0.39 (0.17, 0.57) | 0.24 (0.03, 0.44) |
|
| |||||
| Total walkingd | 0.27 (0.11, 0.41) | 0.32 (0.10, 0.51) | 0.17 (−0.06, 0.38) | 0.40 (0.18, 0.58) | 0.18 (−0.03, 0.39) |
|
| |||||
| Total moderate-PAe | 0.18 (0.02, 0.33) | 0.29 (0.07, 0.48) | 0.04 (−0.19, 0.26) | 0.26 (0.02, 0.46) | 0.10 (−0.12, 0.31) |
|
| |||||
| Moderate-PAf | |||||
| Recreational walking | 0.23 (0.07, 0.37) | 0.31 (0.09, 0.50) | 0.11 (−0.12, 0.33) | 0.33 (0.11, 0.53) | 0.12 (−0.10, 0.32) |
|
| |||||
| Total walking | 0.24 (0.08, 0.38) | 0.32 (0.10, 0.51) | 0.12 (−0.11, 0.33) | 0.37 (0.15, 0.56) | 0.11 (−0.11, 0.32) |
|
| |||||
| Total moderate-PA | 0.19 (0.03, 0.34) | 0.31 (0.09, 0.50) | 0.03 (−0.19, 0.25) | 0.24 (0.01, 0.45) | 0.13 (−0.09, 0.34) |
|
| |||||
| Bout minutesg | |||||
|
| |||||
| Recreational walking | 0.36 (0.21, 0.49) | 0.28 (0.05, 0.47) | 0.43 (0.22, 0.59) | 0.35 (0.12, 0.54) | 0.34 (0.13, 0.52) |
|
| |||||
| Total walking | 0.37 (0.22, 0.50) | 0.31 (0.09, 0.50) | 0.41 (0.20, 0.58) | 0.40 (0.18, 0.58) | 0.31 (0.09, 0.49) |
|
| |||||
| Total moderate-PA | 0.36 (0.21, 0.49) | 0.38 (0.17, 0.56) | 0.33 (0.11, 0.51) | 0.32 (0.09, 0.52) | 0.38 (0.17, 0.55) |
Assessed by modified RESIDE’s Neighborhood Physical Activity Questionnaire
Steps per day
Minutes per day reported walking for recreation
Minutes per day reported total walking
Minutes per day reported total moderate-PA (including walking)
Minutes per day moderate-PA (2020 ≤ counts/min ≤ 5998)
Minutes per day PA ≥ moderate intensity in bouts ≥ 10 min duration allowing 2 min of interruption.
BMI: body mass index; CI: confidence interval; PA: physical activity; SCC: Spearman correlation coefficient
Two correlations differed substantially by BMI category. SCCs were stronger for women with BMI < 37 for accelerometer assessed steps/d (SCC 0.29 vs. 0.04) and moderate-PA (min/d) (SCC 0.31 vs. 0.03) compared with self-reported moderate-PA. These differences remained when the results were re-stratified by obesity class: overweight and obesity class I (BMI < 35, n=45) compared to obesity class II and III (BMI ≥ 35, n=107). For those with BMI < 35, the SCC between accelerometer-measured steps/d and self-reported moderate-PA was 0.31 (95% CI: 0.02, 0.55) versus 0.11 (95% CI: −0.08, 0.29) for the higher BMI group. Similarly, the SCC for accelerometer-measured moderate-PA with self-reported moderate-PA were 0.34 (95% CI: 0.05–0.57) for women with BMI < 35 and 0.12 (95% CI: −0.07, 0.30) for those with BMI ≥ 35.
Three correlations were stronger among unemployed relative to employed participants: accelerometer assessed steps/d compared with self-reported total walking (min/d) (SCC 0.40 vs. 0.18), and accelerometer assessed moderate-PA (min/d) compared with self-reported recreational walking (min/d) (SCC 0.33 vs. 0.12) and total walking (min/d) (SCC 0.37 vs. 0.11).
When stratified by education or race/ethnicity, no meaningful differences were observed (data not shown) in the SCCs for self-report measures (walking for recreation, total walking, and moderate-PA) compared with accelerometer measures (steps/d, moderate-PA, and bouts of moderate to vigorous-PA).
Among those retested (n=57), the modified RESIDE questionnaire demonstrated substantial test-retest reliability for measures of walking for recreation (min/d) (ICC 0.64; 95% CI: 0.46, 0.77), total moderate-PA (min/d) (ICC 0.68; 95% CI: 0.51, 0.80), and total activity (min/d) (ICC 0.67; 95% CI: 0.50, 0.79). The total walking (min/d) measure (ICC 0.56; 95% CI: 0.36, 0.71) had moderate reliability, while the reliability of walking for transportation (min/d) (ICC −0.01; 95% CI: −0.26, 0.25) and vigorous-PA (min/d) (ICC 0.11; 95% CI: −0.15, 0.36) were poor.
Discussion
This study is among the first to evaluate the validity and reliability of a PA questionnaire in an overweight and obese, low-income population. Key domains of the modified RESIDE questionnaire demonstrated fair criterion-related validity with accelerometer measures, while the test-retest reliability of recreational walking and moderate-PA measures was substantial. There were no differences in validity by race/ethnicity or education and modest differences in the validity of some measures by BMI and employment status. The evidence for criterion-related validity and test-retest reliability of walking measures suggest that the modified RESIDE questionnaire was an appropriate instrument to assess the Weight Wise II PA intervention effect on walking.
Current US guidelines call for PA to be accumulated in bouts lasting ten or more minutes. Therefore, an important finding of this study was that the evidence for criterion-related validity was strongest between modified RESIDE questionnaire measures and accelerometer-measured moderate to vigorous-PA in bouts of at least ten minutes (SCC 0.36–0.37). Longer bouts of PA may be more memorable, thus longer duration PA may be reported with greater accuracy and captured more completely by the questionnaire.
Evidence for validity of the modified RESIDE walking measures compared to accelerometer-measured steps/d was also fair. Walking was a core recommendation of the Weight Wise II PA intervention making the validity of these measures important. Cerin and colleagues10 found a stronger correlation between total walking (min/wk) and accelerometer step counts in their study of a modified RESIDE questionnaire (r = 0.48) than identified here (SCC 0.27). However, a greater range of activity levels among participants in their study may account for the stronger criterion-validity relative to our sample: the standard deviation of accelerometer-measured counts/min among older Chinese adults was 107 10 compared to only 62 counts/min in our sample.
Reduced criterion-related validity of PA questionnaires among individuals with higher BMI was reported previously.5,7 One explanation is increased over-reporting of PA among overweight individuals due to social desirability bias.5 In our study, self-reported PA did not differ by BMI; however, participants in the lower BMI group completed more steps/d according to accelerometer data. Although central girth can alter the tilt angle of the accelerometer,23 previous studies indicate that BMI-based variations in device accuracy are unlikely to explain the criterion-related validity differences identified here.24–26 However, studies of device accuracy among individuals in the highest BMI classes (BMI ≥ 35) are needed.
We also identified differences in questionnaire validity by participant employment status. Correlations between accelerometer-measured steps/d and self-reported total walking, and accelerometer-measured moderate-PA compared with total and recreational walking were stronger among unemployed compared to employed participants. Discrepancies in the types of PA captured by the questionnaire and accelerometer may contribute to this difference. Occupational PA by employed participants would have been captured by the accelerometer but not reported on the questionnaire, which did not address occupational activities. Unemployed participants reported more walking, moderate, and total PA, while accelerometer measures (steps/d, light-PA) indicated greater activity among employed participants. Capturing occupational activity, such as walking at work, could improve the validity of the modified RESIDE questionnaire.
Substantial test-retest reliability supports the use of the modified RESIDE questionnaire to detect changes in individuals’ behavior over time, a goal of the Weight-Wise II study. Test-retest reliability of the RESIDE questionnaire and modified versions may be enhanced by assessing usual PA, which may be a more stable measure than PA assessed for a specific seven-day period.6 Our modified questionnaire had similar test-retest reliability to two other modified versions of RESIDE with average time to retest of 1710 and 2.711 days. However, the original questionnaire had higher reliability among a convenience sample of Australian adults (ICC 0.82 to 0.90; 7 day mean test-retest interval).6 The longer test-retest interval in our study (mean 25.5 days) may contribute to weaker ICC estimates. In our sample, poor test-retest reliability of the walking for transportation and vigorous-PA measures are likely attributable to the infrequency of these activities in this population.
This study has several limitations. The results should not be generalized to populations distinct from the overweight-obese, mid-life American women in our sample. Also, due to low PA levels, we were unable to assess variation in criterion-related validity and test-retest reliability of the modified RESIDE questionnaire by PA level. Additionally, the small number of participants (n=57) who completed the retest questionnaire precluded stratification of reliability analyses by BMI, education, race/ethnicity, or employment status. It is unclear from previous studies whether these characteristics impact questionnaire reliability.5,9 Further, the accelerometer count thresholds used to define PA intensity may be higher than are appropriate for the mid-life women in our sample.27 This could result in misclassification of moderate-PA as light-PA and consequently underestimation of moderate-PA from accelerometer data. ActiGraph devices may be less accurate at slow walking speeds (< 2.5 miles per hour)26 and uniaxial accelerometers underestimate non-ambulatory activities such as lifting.28 Slow walking speeds among our participants could contribute to the limited strength of correlations between questionnaire and accelerometer measures. Consistently limited criterion-related validity of PA questionnaires with accelerometer data points to the lack of a definitive gold standard against which to compare PA measures.
Conclusions
We identified fair validity and substantial reliability of a brief, 16-item questionnaire used to assess PA among low-income overweight women, an understudied population facing significant health disparities. The evidence for validity and reliability was similar to that documented for other self-report PA instruments in two systematic reviews4,8 and low-income populations.29,30 However, our data suggest that the modified RESIDE questionnaire may not capture PA as accurately among women with a BMI ≥ 37 kg/m2 and employed women. Modifications that may improve the validity of the questionnaire include additional items to capture occupational and light intensity PA, specifically walking at work, and contextual cues to aid respondents in recalling short-duration PA bouts.
Supplementary Material
Practical Implications.
The modified RESIDE questionnaire rapidly captures self-reported physical activity among overweight and obese women with accuracy similar to that of other self-report instruments.
Most behavioral interventions for low-income women to reduce CVD risk and promote weight loss focus on increasing walking. Based on validity and reliability reported here, the modified RESIDE questionnaire is a reasonable instrument for assessing intervention effect on walking.
Additional items to capture occupational activity may improve the accuracy of questionnaire measures among employed individuals.
Acknowledgments
This study was supported through funding by the Centers for Disease Control and Prevention (CDC) grant #5R18DP001144 and from the University of North Carolina (UNC) Center for Health Promotion and Disease Prevention (a Prevention Research Center) through funding by CDC cooperative agreement #U48/DP000059. The content is solely the responsibility of the authors and does not necessarily represent the official views of the CDC. This study is registered at www.clinicaltrials.gov (No. NCT01141348). We gratefully acknowledge Dr. Ziya Gizlice of the UNC Center for Health Promotion and Disease Prevention for his contribution to the statistical analyses of these data. The lead author further acknowledges the support of the UNC Royster Society of Fellows.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Lee IM, Shiroma EJ, Lobelo F, et al. Effect of physical inactivity on major non-communicable diseases worldwide: An analysis of burden of disease and life expectancy. Lancet. 2012;380(9838):219–29. doi: 10.1016/S0140-6736(12)61031-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Troiano RP, Berrigan D, Dodd KW, et al. Physical activity in the United States measured by accelerometer. Med Sci Sports Exerc. 2008;40(1):181–188. doi: 10.1249/mss.0b013e31815a51b3. [DOI] [PubMed] [Google Scholar]
- 3. [Accessed September 20, 2012];Trust for America’s Health. F as in fat: How obesity threatens America’s future. 2010 http://healthyamericans.org/reports/obesity2010/. Updated 2010.
- 4.Prince SA, Adamo KB, Hamel ME, et al. A comparison of direct versus self-report measures for assessing physical activity in adults: A systematic review. Int J Behav Nutr Phys Act. 2008;5:56. doi: 10.1186/1479-5868-5-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bull FC, Maslin TS, Armstrong T. Global physical activity questionnaire (GPAQ): Nine country reliability and validity study. J Phys Act Health. 2009;6(6):790–804. doi: 10.1123/jpah.6.6.790. [DOI] [PubMed] [Google Scholar]
- 6.Giles-Corti B, Timperio A, Cutt H, et al. Development of a reliable measure of walking within and outside the local neighborhood: RESIDE’s neighborhood physical activity questionnaire. Prev Med. 2006;42(6):455–459. doi: 10.1016/j.ypmed.2006.01.019. [DOI] [PubMed] [Google Scholar]
- 7.Ferrari P, Friedenreich C, Matthews CE. The role of measurement error in estimating levels of physical activity. Am J Epidemiol. 2007;166(7):832–40. doi: 10.1093/aje/kwm148. [DOI] [PubMed] [Google Scholar]
- 8.Helmerhorst HJ, Brage S, Warren J, et al. A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. Int J Behav Nutr Phys Act. 2012;9:103–5868-9-103. doi: 10.1186/1479-5868-9-103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Meyer AM, Evenson KR, Morimoto L, et al. Test-retest reliability of the women’s health initiative physical activity questionnaire. Med Sci Sports Exerc. 2009;41(3):530–538. doi: 10.1249/MSS.0b013e31818ace55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cerin E, Barnett A, Sit CH, et al. Measuring walking within and outside the neighborhood in Chinese elders: Reliability and validity. BMC Public Health. 2011;11:851. doi: 10.1186/1471-2458-11-851. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.McCormack GR, Shiell A, Doyle-Baker PK, et al. Testing the reliability of neighborhood-specific measures of physical activity among Canadian adults. J Phys Act Health. 2009;6(3):367–373. doi: 10.1123/jpah.6.3.367. [DOI] [PubMed] [Google Scholar]
- 12.Braam LA, Ocke MC, Bueno-de-Mesquita HB, et al. Determinants of obesity-related underreporting of energy intake. Am J Epidemiol. 1998;147(11):1081–1086. doi: 10.1093/oxfordjournals.aje.a009402. [DOI] [PubMed] [Google Scholar]
- 13.Johansson G, Wikman A, Ahren AM, et al. Underreporting of energy intake in repeated 24-hour recalls related to gender, age, weight status, day of interview, educational level, reported food intake, smoking habits and area of living. Public Health Nutr. 2001;4(4):919–927. doi: 10.1079/phn2001124. [DOI] [PubMed] [Google Scholar]
- 14.Samuel-Hodge CD, Garcia BA, Johnston LF, et al. Rationale, design, and sample characteristics of a practical randomized trial to assess a weight loss intervention for low-income women: The Weight-Wise II program. Contemp Clin Trials. 2012;33(1):93–103. doi: 10.1016/j.cct.2011.08.009. [DOI] [PubMed] [Google Scholar]
- 15.Samuel-Hodge CD, Garcia BA, Johnston LF, et al. Translation of a behavioral weight loss intervention for mid-life, low-income women in local health departments. Obesity (Silver Spring) 2013;21(9):1764–73. doi: 10.1002/oby.20317. [DOI] [PubMed] [Google Scholar]
- 16.Kraschnewski JL, Keyserling TC, Bangdiwala SI, et al. Optimized probability sampling of study sites to improve generalizability in a multisite intervention trial. Prev Chronic Dis. 2010;7(1):A10. [PMC free article] [PubMed] [Google Scholar]
- 17.Abel MG, Hannon JC, Sell K, et al. Validation of the Kenz Lifecorder EX and ActiGraph GT1M accelerometers for walking and running in adults. Appl Physiol Nutr Metab. 2008;33(6):1155–1164. doi: 10.1139/h08-103. [DOI] [PubMed] [Google Scholar]
- 18.Kozey-Keadle S, Libertine A, Lyden K, et al. Validation of wearable monitors for assessing sedentary behavior. Med Sci Sports Exerc. 2011;43(8):1561–1567. doi: 10.1249/MSS.0b013e31820ce174. [DOI] [PubMed] [Google Scholar]
- 19.Matthews CE, Chen KY, Freedson PS, et al. Amount of time spent in sedentary behaviors in the united states, 2003–2004. Am J Epidemiol. 2008;167(7):875–881. doi: 10.1093/aje/kwm390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Higgins PA, Straub AJ. Understanding the error of our ways: Mapping the concepts of validity and reliability. Nurs Outlook. 2006;54(1):23–29. doi: 10.1016/j.outlook.2004.12.004. [DOI] [PubMed] [Google Scholar]
- 21.Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–428. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
- 22.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 23.DiNallo JM, Downs DS, Le Masurier G. Objectively assessing treadmill walking during the second and third pregnancy trimesters. J Phys Act Health. 2012;9(1):21–28. doi: 10.1123/jpah.9.1.21. [DOI] [PubMed] [Google Scholar]
- 24.Crouter SE, Schneider PL, Bassett DR., Jr Spring-levered versus piezo-electric pedometer accuracy in overweight and obese adults. Med Sci Sports Exerc. 2005;37(10):1673–1679. doi: 10.1249/01.mss.0000181677.36658.a8. [DOI] [PubMed] [Google Scholar]
- 25.Feito Y, Bassett DR, Tyo B, et al. Effects of body mass index and tilt angle on output of two wearable activity monitors. Med Sci Sports Exerc. 2011;43(5):861–866. doi: 10.1249/MSS.0b013e3181fefd40. [DOI] [PubMed] [Google Scholar]
- 26.Feito Y, Bassett DR, Thompson DL. Evaluation of activity monitors in controlled and free-living environments. Med Sci Sports Exerc. 2012;44(4):733–741. doi: 10.1249/MSS.0b013e3182351913. [DOI] [PubMed] [Google Scholar]
- 27.Evenson KR, Buchner DM, Morland KB. Objective measurement of physical activity and sedentary behavior among US adults aged 60 years or older. Prev Chronic Dis. 2012;9:E26. [PMC free article] [PubMed] [Google Scholar]
- 28.Corder K, Brage S, Ekelund U. Accelerometers and pedometers: Methodology and clinical application. Curr Opin Clin Nutr Metab Care. 2007;10(5):597–603. doi: 10.1097/MCO.0b013e328285d883. [DOI] [PubMed] [Google Scholar]
- 29.Buchowski MS, Matthews CE, Cohen SS, et al. Evaluation of a questionnaire to assess sedentary and active behaviors in the southern community cohort study. J Phys Act Health. 2012;9(6):765–775. doi: 10.1123/jpah.9.6.765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Wolin KY, Heil DP, Askew S, et al. Validation of the international physical activity questionnaire-short among blacks. J Phys Act Health. 2008;5(5):746–760. doi: 10.1123/jpah.5.5.746. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.