Traditional Physical Activity Indexes Derived from the Harvard Alumni Activity Survey Have Low Construct Validity in a Lower Income, Urban Population

Abstract

The purpose of this study is to investigate the construct validity of the Harvard Alumni Activity Survey (HAAS) in an urban, lower income population. Data were collected from 192 smokers enrolled in an antioxidant micronutrient trial. Activity data were compared to body mass index (BMI), diastolic, and systolic blood pressure. The traditional physical activity index (PAI), using data on stair climbing, walking, and sports, was calculated including and excluding body mass. A new scale, the total weekly activity (TWA) scale, was derived from other questions on the HAAS. The PAI scale calculated with body mass was unassociated with BMI and blood pressure. The PAI scale calculated without body mass was unassociated with BMI and systolic blood pressure but was associated with diastolic blood pressure (Beta = −0.001, p = 0.03). The TWA scale was associated with BMI (Beta = −0.01, p = 0.01), diastolic (Beta = −0.03, p = 0.01), and systolic blood pressure (Beta = −0.04, p = 0.01). A one standard deviation change in the TWA scale is predicted to be equivalent to a change of 0.99 BMI units, 2.97 mmHg of diastolic blood pressure, and 3.96 mmHg of systolic blood pressure. This work suggests that the TWA scale has greater construct validity than the traditional PAI scale in this population.

INTRODUCTION

Epidemiologic studies of cardiovascular disease risk, obesity, diabetes, and cancer risk commonly assess physical activity using questionnaire instruments, and there is a wide array of available questionnaires.^1–6 However, available questionnaires were not designed and validated in urban, multiethnic, lower socioeconomic populations. Much of the validity literature has used middle-class, Caucasian study subjects, whereas only a few studies have focused on other racial and ethnic groups.^7–9 In addition, as activity patterns appear to vary quantitatively and qualitatively by socioeconomic class, the lack of validity studies in lower socioeconomic classes is a concern.^10–12 This situation presents challenges when choosing a questionnaire for epidemiologic research on physical activity in urban areas. In our New York City-based intervention study of antioxidant micronutrients among cigarette smokers, we used the Harvard Alumni Activity Survey (HAAS), and here, we report on our efforts to gauge the questionnaire’s validity in this population.¹³

The HAAS is a well-known questionnaire that is short, self-administered, and focuses on leisure time physical activity.⁶ Because there is little data on the validity of physical activity questionnaires in populations similar to ours, these logistical aspects of the HAAS were the reason it was selected. The questionnaire comprises eight main questions with multiple subquestions. Three of the questions regarding walking, climbing stairs and leisure time sports, and exercise activities are used to generate an estimate of energy expenditure through physical activity, referred to as the physical activity index (PAI).^1,14 This traditional PAI scale has been validated against measures of BMI and high density blood lipid levels, activity monitors, maximal oxygen capacity, energy intake estimates, and other questionnaires.^7,8,14–17 Other elements of the questionnaire on the number of hours per day spent sleeping and engaged in vigorous, moderate, light, and sitting activities, which are similar to those used on the international physical activity questionnaire (IPAQ), offer further possibilities for scale development.¹⁸

We have administered the HAAS to a multiethnic cohort of lower income smokers residing in New York City. Because there is little or no literature on the validity of the HAAS in this type of population, we have assessed the construct validity of the traditional PAI scale and a new scale derived from the HAAS that is similar to the IPAQ. In this paper, we report on the associations between physical activity estimates derived from the HAAS and data on body mass index (BMI), and systolic and diastolic blood pressure.

MATERIALS AND METHODS

Study Population

Three hundred and nine committed smokers were enrolled into a randomized clinical trial to determine whether supplementation with vitamins C and E reduces the levels of carcinogen-DNA adducts and oxidative stress.¹³ All participants provided informed consent, and the study was approved by the Herbert Irving Comprehensive Cancer Center and the Columbia Presbyterian Medical Center (CPMC). Study subjects were men and women aged 18–74 years old, who smoked more than ten cigarettes per day at enrollment and answered a recruitment advertisement for the clinical trial. Study subjects were randomized to 15 months of treatment or placebo, and then, all subjects were given placebo for 9 months. Study subjects visited CPMC every 3 months to complete study questionnaire instruments and to donate blood samples for biomarker analyses. At baseline, study subjects were interviewed on their demographic characteristics and smoking history and had their blood pressure taken. At each 3-month visit, the study subjects provided updated information on their recent smoking behavior, had their blood pressure measured, and donated a blood sample in which the nicotine metabolite cotinine was measured. At the 12-month visit, 194 study subjects completed the HAAS questionnaire on their physical activity levels.^6,19 At this time, roughly half the subjects were on treatment, and half were on placebo, and the level of smoking had declined with a substantial portion of the subjects now reporting less than ten cigarettes consumed per day. At enrollment, study subjects completed a questionnaire of medication usage, and two subjects reported using medications for high blood pressure and were excluded from analyses. Table 1 provides descriptive statistics for the subjects who filled out the HAAS questionnaire.

TABLE 1.

Demographic characteristics, BMI, and blood pressure of the study population

	Female (N = 87)	Male (N = 105)	P value for difference by gender
Agea	37.9 (10.12)	39.9 (10.23)	0.19
Ethnicityb			0.12
African American	N = 47	N = 42
Caucasian	N = 29	N = 39
Hispanic	N = 8	N = 18
Other	N = 2	N = 5
Incomeb			0.85
<$10,000	N = 37	N = 42
$10,000–20,000	N = 16	N = 24
$21,000–30,000	N = 16	N = 21
$31,000–40,000	N = 8	N = 10
$41,000+	N = 9	N = 7
Educational attainmentb			0.96
Less than high school	N = 9	N = 11
High school graduate	N = 26	N = 28
Some college	N = 34	N = 43
College graduate	N = 13	N = 15
Post graduate	N = 4	N = 7
Cigarettes smoked per day at 12 months of follow-upb, c			0.12
≤10	N = 38	N = 32
11–15	N = 10	N = 24
16–20	N = 21	N = 27
>20	N = 17	N = 21
Systolic blood pressurea	118.7 (17.63)	125.1 (15.17)	0.01
Diastolic blood pressurea	77.5 (12.47)	81.6 (11.32)	0.02
BMIa	27.4 (7.18)	26.1 (4.63)	0.14

^aMean (SD), P values by t test

^bP value by chi-square

^cAll subjects smoked more than ten cigarettes at enrollment, but there was some decline in cigarettes smoked per day over the course of 1 year of follow-up in the cohort.

Physical Activity Variables

Data from the physical activity questionnaire were processed to generate three activity scales. Although the PAI is always calculated using the questions on walking, climbing stairs and sports, and leisure activity, the exact calculations vary across publications.^1,5,6,14,20 The first calculation of the PAI scale included body mass in kilograms to generate an estimate of energy expenditure in units of kcal/week and is referred to as PAI-1.^5,20 A second PAI scale (PAI-2) was calculated without including mass and is equivalent to kcal/week term used by Paffenbarger et al.^1,14 A second scale referred to as total weekly activity (TWA) was developed from the HAAS question on daily activities in the past year. This question collects data on the number of hours in a typical weekday and weekend day subjects spent sleeping, engaged in quiet sitting activity, in light activity, moderate activity, or vigorous activity. The recorded hours of activity in each category, for each day, was a total of 24 h. Vigorous activity was assigned a weight of 7 metabolic equivalents (MET), moderate activity 4.5 MET, light activity 3 MET, sitting activity 1 MET, and sleeping 0.9 MET.^6,21 A MET score expresses the metabolic rate required to conduct an activity compared to the resting metabolic rate.²¹ To estimate total weekly activity, the MET × h score generated from the question on weekday activity was multiplied by 5, and the MET × h score for weekend day activity was multiplied by 2, and the resulting quantities were summed. Thus, the TWA scale is expressed in units of MET × h per week. Sufficient data was available to calculate the PAI-1 scale for 191 subjects, the PAI-2 scale for 192 subjects, and the TWA scale for 183 subjects.

Statistical Analyses

The physical activity scales were analyzed to determine whether estimated activity levels varied by demographic factors. PAI and TWA data were analyzed for associations with demographic variables using t tests and analysis of variance. The PAI scales had a log normal distribution and were log_e transformed for analyses with demographic variables, and geometric means and corresponding standard deviations are reported.²²

For initial analyses of physiological parameters and the PAI and TWA scales, quartile categories were created for the physical activity scales and the mean BMI, and diastolic and systolic blood pressure was calculated for each quartile. Multivariate analyses were also performed to determine whether each activity scale was associated with the physiological parameters after control for possible confounding factors. Separate linear regression analyses were performed with blood pressure and BMI as the dependent variables and each physical activity scale as the independent variable. The activity scales were entered into the models as continuous variables. These regression models controlled for gender, age, ethnicity (Caucasian, African American, Hispanic, and Other coded with dummy variables), and cigarettes smoked per day at the 12-month time point. Each scale was analyzed separately for associations with BMI, systolic, and diastolic blood pressure. Blood pressure data were available from 171 subjects at the 12-month time point. BMI was calculated using the height measured at enrollment into the cohort and mass measured at the 12-month time point. If data were not available from the 12-month time point, mass measured at the nearest available time point was used.

RESULTS

Data on the PAI-1, PAI-2, and TWA indexes of physical activity are shown in Table 2. Both the PAI-1 and PAI-2 measures were associated with age and gender but not race/ethnicity. The TWA scale was associated with gender and race/ethnicity but not age. None of the physical activity scales were associated with cigarettes smoked per day or with blood cotinine levels measured at the 12-month visit.

TABLE 2.

Physical activity levels by demographic variables

	PAI-1 (kcal/week) Geometric mean (geometric standard deviation) N	PAI-2 (kcal/week) Geometric mean (geometric standard deviation) N	TWA (MET × h/week) mean (standard deviation) N
Overall	1,786 (2.71) 189	1,721 (2.50) 190	410 (99) 181
Age (years)
<32.25	2,382 (2.52) 47	2,329 (2.33) 48	459 (107) 47
> = 32.25–<39.00	2,140 (3.03) 44	1,957 (2.78) 44	477 (125) 40
> = 39.00–<45.00	1,627 (2.51) 44	1,585 (2.29) 44	452 (121) 41
> = 45.00	1,296 (2.56) 54	1,267 (2.37) 54	393 (116) 53
	p = 0.01	p = 0.01	p = 0.39
Gender
Male	2,258 (2.43) 104	2,086 (2.24) 104	468 (130) 101
Female	1,342 (2.87) 85	1,364 (2.68) 86	433 (119) 82
	p = 0.001	p = 0.001	p = 0.06
Ethnicity
African American	1,707 (2.85) 89	1,639 (2.61) 89	480 (136) 86
Caucasian	1,703 (2.59) 67	1,689 (2.42) 68	384 (88) 65
Hispanic	2,185 (2.56) 26	1,997 (2.47) 26	464 (131) 23
Other	2,370 (2.27) 7	2,216 (2.12) 7	414 (53) 7
	p = 0.58	p = 0.69	p = 0.03
Cigarettes smoked per day at 12 months
≤10	1,919 (2.85) 70	1,862 (2.63) 70	412 (100) 67
11–15	1,958 (2.76) 33	1,909 (2.58) 34	413 (101) 32
16–20	1,958 (2.67) 48	1,841 (2.40) 48	406 (95) 46
>20	1,286 (2.39) 38	1,246 (2.21) 38	409 (104) 36
	p = 0.16	p = 0.11	p = 0.99
Blood cotinine at 12 months (ng/ml)
≤143	2,080 (2.97) 43	2,066 (2.71) 44	393 (111) 42
144–222	1,525 (2.83) 44	1,408 (2.55) 44	423 (88) 42
223–313	1,709 (2.35) 44	1,669 (2.16) 44	385 (98) 41
>313	1,835 (2.77) 43	1,751 (2.57) 43	435 (99) 42
	p = 0.50	p = 0.28	p = 0.07

The analyses of physiological parameters indicate that the traditionally calculated PAI scales have poor construct validity in this population but the TWA scale has better construct validity. Table 3 shows the mean BMI, diastolic, and systolic blood pressure by quartiles of the PAI and TWA scales. The PAI measures were not associated with BMI or either of the measures of blood pressure. The TWA measure was associated with each of the physiological parameters, although in these univariate analyses, the associations were of borderline statistical significance. Table 4 shows the linear regression and partial R coefficients from multivariate models of the association between the activity scales and the physiological parameters. After control for potential confounding by age, gender, cigarettes smoked per day at 12-months, and race the PAI-1 scale was not associated with any of the physiological parameters, and the PAI-2 scale was weakly but significantly associated with diastolic blood pressure only. However, after control for the potential confounding factors, the TWA scale was significantly inversely associated with BMI, diastolic, and systolic blood pressure. Because blood pressure is sensitive to cigarette smoking, all analyses of blood pressure were repeated with additional control for blood cotinine levels at the 12-month visit, cigarettes smoked per day at enrollment, and the average number cigarettes smoked per day between enrollment and the 12-month time point. The results of these analyses were essentially the same as those reported in Table 4. As sufficient data were not available to calculate the PAI and TWA scales for everyone, and there were different sample sizes for each of the scales, multivariate analyses were repeated on the subset of subjects from whom data on all three scales were available. The results from these analyses were essentially the same as reported above and in Table 4.

TABLE 3.

Physiological parameters by level of activity

Physical activity scale	BMI mean (standard deviation) N	Diastolic blood pressure mean (standard deviation) N	Systolic blood pressure mean (standard deviation) N
PAI-1 (kcal/week)
<910.00	28.12 (5.54) 46	82.50 (11.42) 40	128.88 (18.86) 40
910.00–<1,680.00	24.65 (4.88) 48	78.53 (11.81) 43	118.23 (12.20) 43
1,680.00–<3,431.85	26.36 (5.14) 48	77.75 (11.83) 40	120.38 (17.85) 40
> = 3,431.85	27.78 (7.41) 47	79.36 (13.03) 43	121.16 (16.01) 43
	P for trend = 0.28	P for trend = 0.60	P for trend = 0.36
PAI-2 (kcal/week)
<980.00	28.25 (5.49) 46	82.74 (11.68) 38	129.47 (19.16) 38
980.00–<1,660.73	24.81 (4.81) 49	79.24 (11.39) 44	119.20 (12.24) 44
1,660.73–<3,056.94	27.44 (7.42) 48	78.27 (12.15) 41	120.37 (17.43) 41
> = 3,056.94	26.41 (5.26) 46	78.08 (12.77) 44	120.05 (16.02) 44
	P for trend = 0.64	P for trend = 0.17	P for trend = 0.11
TWA (MET × h/week)
<363.80	27.95 (7.76) 45	79.64 (13.19) 39	124.41 (15.95) 39
363.80–<441.80	27.09 (5.79) 44	82.26 (13.36) 40	123.13 (19.25) 40
441.80–<515.50	26.88 (4.56) 46	80.41 (11.05) 41	123.83 (17.29) 41
> = 515.50	25.17 (5.44) 45	75.44 (10.68) 39	117.26 (14.05) 39
	P for trend = 0.03	P for trend = 0.07	P for trend = 0.06

TABLE 4.

Multivariate^a linear regression analyses of physiological parameters and physical activity scales

Physical activity scale	BMI	Diastolic blood pressure	Systolic blood pressure
PAI-1 (kcal/week)	Beta = 0.000, Partial R = 0.07, p = 0.41	Beta = 0.000, Partial R = −0.11, p = 0.15	Beta = 0.000, Partial R = −0.03, p = 0.67
PAI-2 (kcal/week)	Beta = 0.000, Partial R = −0.07, p = 0.32	Beta = −0.001, Partial R = −0.17, p = 0.03	Beta = −0.001, Partial R = −0.09, p = 0.29
TWA (MET × h/week)	Beta = −0.01, Partial R = −0.20, p = 0.01	Beta = −0.03, Partial R = −0.23, p = 0.01	Beta = −0.04, Partial R = −0.22, p = 0.01

^aControlling for age, gender, ethnicity, and cigarettes smoked per day at 12 months

DISCUSSION

The HAAS has been used to investigate the health outcomes associated with an active lifestyle and has shown associations between activity and reduced risk for cancer and cardiovascular disease.^2,5 The validity of the traditional scales derived from the HAAS has been tested in a number of settings but not extensively in multiethnic, low income, urban populations.⁸ As in other studies, physiologic parameters, BMI, systolic, and diastolic blood pressure, known to be affected by physical activity were used as criteria against which to assess the construct validity of the traditional PAI scale in our study population.^15,23–28 The results show that after control for confounding factors, the traditional PAI scales were generally not associated with these parameters in this population and thus probably have low construct validity in this population. However, the TWA scale does have construct validity, and a one standard deviation change in the TWA scale score is equivalent to a predicted difference of 0.99 units for BMI, −3.96 mmHg in systolic blood pressure, and −2.97 mmHg in diastolic blood pressure.

The populations used in past studies showing the validity of the HAAS have often not been well described but appear to be predominantly Caucasian and middle or upper middle class. The Study of Activity, Fitness and Exercise (SAFE) validated the HAAS in a population recruited from the University of Minnesota community, which was 94% Caucasian, highly educated, and predominantly employed in administrative or professional positions.^7,14 A second large validation study that included the HAAS was conducted in a population described as being mostly comprised of employees at the US Department of Agriculture.¹⁷ Similarly, another validity study of the HAAS was conducted in a population described only as “hospital employees,” and mean PAI results were not reported.¹⁶ A large, well-described study of physiological correlates of the HAAS was conducted in the Boston Metropolitan area and recruited study subjects from areas with below average, average, and above average household incomes.^15,29 However, the results were not shown by income defined areas, and income was not included in the multivariate models, so the role of socioeconomic class is unclear in this study.¹⁵ Lastly, evidence for the validity of the HAAS was found in a study of Latinos.⁸ This population is perhaps the closest to ours, although they would appear to have a higher socioeconomic status as 70% had a family income over $25,000/year, and 98% had completed high school.⁸ Compared to the PAI-2 data presented here, with a mean energy expenditure of 1,721 kcal/day, these studies have found similar or slightly higher average energy expenditures: 1,736 kcal/week in the SAFE study, 2,049 kcal/week in the Department of Agriculture Study, and 1,805 kcal/week in the Boston study. Clearly, past validity studies have used study populations substantially different from the population studied here and have not considered how socioeconomic class may influence validity.

The PAI scale reflects, to a large degree, leisure time activity; however, past studies suggest that individuals of lower socioeconomic status are less likely to engage in leisure time activity and are more likely to have higher levels of occupational and home activity.^10–12 The lack of association between the PAI scale and physiological parameters is probably because of the study subjects being active in domains not measured by the scale.

The TWA scale was derived using questions on weekday and weekend hours spent asleep, engaged in sitting, light, moderate, and vigorous activity, and the TWA scale is conceptually similar to the IPAQ. These questions are framed to reflect the past year of activity, and the examples provided for each level of activity include household, occupational, and leisure time activities. Our multivariate analyses show that this scale is significantly associated with all three of the physiological measures. A comparison of the content covered by the two scales and their relations to the physiological parameters suggests that occupational and household activities are important contributors to activity in this population. This interpretation is consistent with past literature on activity patterns in lower socioeconomic groups and non-Caucasians.^10–12,30

Prior national surveys have shown recreational activity levels to decline with age and men to have higher activity levels than women.³¹ These trends were reflected in our data although more so with the traditional PAI scale than the TWA scale, which is consistent with the PAI scale focusing on leisure time activity. We did not find activity estimates calculated by the traditional PAI scale to vary significantly by race/ethnicity, but the Caucasians scored lower on the TWA scale than Hispanics or African Americans. Prior representative surveys have almost entirely been confined to measuring leisure time physical activity and have shown Caucasians to have higher levels than African Americans.³¹ Thus, the types of activity measured in these national surveys are most comparable to the traditional PAI scale, which was not associated with race/ethnicity in our population. It is possible that the associations between leisure time activity and racial/ethnicity observed in the overall population do not hold in lower socioeconomic groups such as our study population. The 1990 National Health Interview Survey, a representative survey that did assess occupational activity, found that African Americans had higher levels of occupational activity than Caucasians.³² Thus, the higher TWA scores for African Americans in our study could reflect higher occupational activity in this group.

The use of physiological parameters in validity studies reflects an analysis of construct validity.^28,33 The construct validity of a measurement is defined as the extent to which the measure correlates with other factors or measures known to correlate with the underlying construct that the measurement of interest is supposed to reflect.³³ Physical activity is known to influence body size and BMI, and past validity studies have used measures of BMI and adiposity as validation criteria.^7,15,28 Physical activity and exercise are known to reduce blood pressure and are prescribed as a treatment for hypertension, and past validity studies have used blood pressure as validity criterion^{23–25,27,28}. As such, analyses of the association between measured physical activity levels and BMI and blood pressure serve as a good test of the construct validity of the physical activity questionnaire.²⁸ However, as these physiological measures are not a gold standard measure of physical activity, these analyses do not constitute an assessment of criterion validity, which is a limitation of this study.³³

Although these physiological measures have been used in past validity studies, their use raises several issues that should be considered. The first is that high blood pressure is a commonly treated condition. At enrollment, the study subjects were asked about the use of blood pressure medication, and subjects that were using blood pressure medication were excluded from the analyses, however these questions were not included in the follow-up interviews. It is possible that, in the 12 months of follow-up, some additional study subjects began treatment for high blood pressure. However, this study population appears to be quite medically underserved. Of the 40 subjects whose systolic blood pressure was higher than 140 or diastolic blood pressure was greater than 90, only two subjects reported taking blood pressure medication. Thus, we do not think it likely that many of the study subjects began taking medication during the 12-month follow-up. The second consideration is that increased BMI and elevated blood pressure are chronic conditions that probably developed over a number of years. Conceptually, activity levels engaged in over a period of multiple years should be most strongly associated with the physiological parameters; yet, the HAAS only captures past year activity data. However, any misspecification of the appropriate timeframe applies all of the scales used here and is unlikely to explain why the TWA scale is strongly associated with the physiological parameters and the PAI-1 and PAI-2 scales are not.

One further limitation of this research is that it was restricted to a study population of smokers. Confounding by extent of smoking does not appear to explain our results. The results were consistent after controlling for a host of smoking-related questionnaire variables and after control for blood cotinine, a biomarker of smoking. That the subjects were all smokers may limit the generalizability of our results but only if smoking alters how activity is partitioned across domains of activity. For instance, if household or occupational activity were less important sources of physical activity in low-income nonsmokers than low-income smokers, the TWA scale would likely perform differently in nonsmokers compared to smokers. However, whereas smoking may be associated with the overall extent of activity, we do not think it is likely that smoking alters how activity is partitioned across domains of activity.

The USA faces an epidemic of obesity, and increases in obesity and sedentary lifestyles are likely to lead to increases in cardiovascular disease, diabetes, and certain cancers. However, the data presented here and the observation that most prior research to validate physical activity questionnaires has occurred in wealthier, suburban populations suggests that current research tools may not be adequate for surveillance and research on activity patterns in the urban environment. This represents a health disparity that needs to be addressed. Existing questionnaires should be studied for their validity in the urban environment, and new questionnaires and survey tools may need to be designed. The traditional PAI scales appear to have poor construct validity in our study population; however, the scale we created that includes measures of occupational and household activity does appear to have construct validity for measuring physical activity. These results highlight the need for validity studies that are more demographically varied and consider stratification by race/ethnicity and socioeconomic status.

Abbreviations

BMI

body mass index

HAAS

Harvard alumni activity survey

PAI

physical activity index

TWA

total weekly activity