A Community Based Study to Test the Reliability and Validity of Physical Activity Measurement Techniques

Abstract

Introduction:

Physical activity (PA) is protective against non-communicable diseases and it can reduce premature mortality. However, it is difficult to assess the frequency, duration, type and intensity of PA. The global physical activity questionnaire (GPAQ) has been developed by World Health Organization with the aim of having valid and reliable estimates of PA. The primary aim of this study is to assess the repeatability of the GPAQ instrument and the secondary aim is to validate it against International Physical Activity Questionnaire (IPAQ) and against an objective measure of PA (i.e., using pedometers) in both rural and peri-urban areas of North India.

Methods:

A total of 262 subjects were recruited by random selection from Ballabgarh Block of Haryana State in India. For test retest repeatability of GPAQ and IPAQ, the instruments were administered on two occasions separated by at least 3 days. For concurrent validity, both questionnaires were administered in random order and for criterion validity step counters were used. Spearman's correlation coefficient, intra-class correlation (ICC) and Cohen's kappa was used in the analysis.

Results:

For GPAQ validity, the spearman's Rho ranged from 0.40 to 0.59 and ICC ranged from 0.43 to 0.81 while for IPAQ validity, spearman correlation coefficient ranged from 0.42 to 0.43 and ICC ranged from 0.56 to 0.68. The observed concurrent validity coefficients suggested that both the questionnaires had reasonable agreement (Spearman Rho of >0.90; P < 0.0001; ICC: 0.76-0.91, P < 0.05).

Conclusions:

GPAQ is similar to IPAQ in measuring PA and can be used for measurement of PA in community settings.

INTRODUCTION

There is ample evidence that physical activity (PA) is protective against various diseases and it can reduce premature mortality. PA level is strongly and inversely associated with the risk of cardiovascular morbidities and premature mortality.[1,2] Regular physical exercise improves insulin sensitivity in lifestyle-related diseases like type 2 diabetes and hyperlipidemia.[3,4,5] Further, continued PA is associated with a higher energy turnover, with important implications for the transport, storage and utilization of lipid fuels. Data from the lipid clinics prevalence study revealed that those participants, irrespective of the sex, who reported some “strenuous” PA had higher high-density lipoprotein cholesterol levels than those who reported none.[6] PA has also been linked to reduced risk of metabolic syndrome.[7] Thus, these findings underscore the importance of PA for prevention of chronic diseases and early mortality due to these diseases.

PA is a complex, multidimensional behavioral factor and it is difficult to assess the frequency, duration, type and intensity of PA. Instruments to capture PA differ in their structure, question order and wording. There is no standardized approach to measurement of PA; thus, comparisons across studies and especially internationally, are difficult.[8,9,10,11]

For various epidemiologic studies, good measurement of PA is needed for relating it to both potential predictors and a variety of health outcomes. Aside from self-reported measures, various devices including step counters may be used for the purpose of assessment.[12] Population surveillance systems require a robust, short set of questions to assess the pattern of PA. The global physical activity questionnaire (GPAQ) was developed by World Health Organization (WHO) for PA surveillance in countries. It collects information on PA participation as well as sedentary behavior. This instrument was mainly developed for use in developing countries.[13]

The major strengths of GPAQ include the fact that it is domain specific, which implies that it assesses different types of PA undertaken in three domains plus sitting. The three domains include: Activity at work, travel to and from places and recreational activities. Furthermore, it quantifies exposure apart from the added advantage of cross cultural application.[14]

With an increased interest in chronic diseases and in the role of PA to prevent the same, along with the fact that there is a lacuna in data for inter-country comparisons, there is a need for standardized measurements of PA.[13] Compared to the International Physical Activity Questionnaire (IPAQ), which has an acceptable level of validity and reliability, GPAQ required extensive testing before it could be used for non-communicable disease risk factor surveillance.[8]

In order to test its psychometric properties, WHO initiated a multi-site validation study. In India, Ballabgarh, which was one of the sites for pre-testing the steps approach, was involved in this activity. This paper reports the result of the validation exercise undertaken in Ballabgarh. The full report has earlier been documented.[15]

As a part of the process of validation of GPAQ, we assessed the repeatability of the GPAQ instrument, validated it against IPAQ, an internationally acceptable measurement of PA and against an objective measure of PA (i.e., using pedometers) in both rural and peri-urban areas of India.

METHODS

Study site and study subjects

This study was conducted in North India in Ballabgarh area of Faridabad district, around 25 miles from New Delhi. The age range of the participant for this study purpose was 15-65 years. Participants were recruited from both peri-urban and rural areas. Both male and female subjects were recruited from different areas by random selection to represent different socio-economic and educational backgrounds.

Sample size and sampling method

In peri-urban area we selected three colonies randomly, after dividing colonies in 2 groups - slum colonies and other colonies. We randomly selected 1 Slum and 2. Other colonies in peri-urban area. In a rural area, two villages (Atali and Chandawali) were selected randomly from a list of twenty eight villages under All India Institute of Medical Sciences (AIIMS) run Comprehensive Rural Health Services Project (CRHSP). At both the places we picked first house randomly and then every 4^th house was included. Our aim was to recruit 240 subjects, 120 each from rural and peri-urban area.

Study instruments

IPAQ was used in the current study. This tool was developed for measuring PA in 1998 and was followed by extensive reliability and validity testing undertaken across 12 countries (14 sites) during 2000. The final results suggested that this tool had acceptable measurement properties for use in many settings and in different languages and was suitable for national population-based prevalence studies of participation in PA. The short version of IPAQ tool used in the current study. The tool had a set of 4 questionnaires. Long (5 activity domains asked independently) and short (4 generic items) versions, which were self-administered. The questions asked about the time spent being physically active in the last 7 days. The tool was administered in local language (i.e., Hindi) after pre-testing.

GPAQ was developed by WHO and used in Steps for measurement of PA. The GPAQ covers several components of PA, such as intensity, duration and frequency and it assesses three domains in which PA is performed (occupational PA, transport-related PA and PA during discretionary or leisure time). The tool was administered in local language (i.e., Hindi) after pre-testing.

To validate GPAQ against an objective measure, we used pedometers, which is a simple device to assess PA. Pedometers are a useful instrument for objectively assessing PA and are found to provide a valid and reliable measure of ambulatory activity, which is most prevalent type activity in today's life.[12,16] Step counters (Yamaha SW 700) were used for counting steps. Digital weighting (Seca) machine and stadiometer (Seca) were used to record weight and height respectively.

Data collection

Data collection was done using household interviews. Each person was contacted thrice during the study [Table 1]. On the first occasion, general questions were asked and height and weight recorded. Persons were informed about the study and informed consent was obtained before giving training in the use of step counters. GPAQ and IPAQ questionnaire were administered in random order; half completed GPAQ first and half completed IPAQ first. All interviews were conducted in winters i.e., in the months of December to February and as such there was not much change in weather conditions during the study. All the four workers and one supervisor were trained by the investigator before sending them for data collection into field.

Table 1.

Activities undertaken during 1^st, 2^nd and 3^rd contacts with the study participants

There was an interval of at least 7 days between 1^st and 2^nd visit and of 3-7 days between 2^nd and 3^rd visit. During the process of progress review, it was noted that the participants were reporting verbatim the same activities that they reported 3 days back. This indicated that they had complete recall of the previous interview. This resulted in an identical assessment of PA at both the occasions.

However, as this was a part of multi-site study this was not changed and we continued to collect the data in the same format. However, in order to assess reliability we repeated this question after 1 month during which they would have definitely forgotten their response to the previous interview. Criterion validity was assessed using step Counters. The study was approved by the Ethical Committee of AIIMS.

Data analysis

Data were entered into MS Excel and were analyzed using SPSS 14.0 (Chicago, IL, USA). Spearman correlation coefficient was used to assess the correlation. For test retest repeatability of GPAQ and IPAQ, the instruments were administered on the two occasions (i.e., visits 2 and 3) separated by at least 3 days. The test for reliability was applied on visit 2 and 3 as per WHO guidelines to have uniformity at all centers. Test-retest repeatability was assessed using intra-class correlation (ICC) for continuous variables and Cohen's kappa for categorical variables.

Metabolic equivalent (MET) values were computed using Ainsworth et al. Compendium of Physical activities: An update of activity codes and MET intensities.[17] Compendiums of PA were developed by Ainsworth et al. to facilitate the coding of PA. We used following MET for various activities as given below for our study purpose:

• Moderate PA (in work or leisure domain) =4.0 METs

• Vigorous PA (in work domain or leisure) =8.0 METs

• Transport related (walking cycling) =3.3 METs.

MET: Defined as the ratio of metabolic rate to resting metabolic rate. 1 MET is considered a resting metabolic rate obtained during quite sitting, it may range from 0.9 MET for sleeping to 18 MET for running at 10.9 mph.[17,18]

Spearman's coefficient (Rho) was applied to document the concurrent validity and criterion validity. Body mass index (BMI) (kg/m²) was used to classify obesity status as per WHO classification.[19,20]

RESULTS

Demographic profile

We interviewed 262 subjects at our study site of which 11 subjects from peri-urban and 17 from a rural area were rejected and not included into the analysis, as their forms were incomplete. Thus, a total of 234 subjects were studied. Out of this, 114 (48.7%) were from peri-urban area and 120 (51.3%) were from rural areas [Table 2]. Both sexes represented well in the study as there were almost equal numbers of male (49.6%) and female (50.4%) participants. Distribution of subjects in different age groups was also well represented (ranged between 14% and 21%), except in the age group 65-74 years (5.6%). About 30% of the participants were either overweight or obese and 13% were underweight as calculated with BMI using WHO criterion [Table 2].

Table 2.

Demographic characteristic of the participants included in the study (N=234)

Reliability

We noted 100% agreement in GPAQ and IPAQ on follow-up visits at both the occasions i.e., during 2^nd and 3^rd visit. Further, we decided to repeat GPAQ and IPAQ questionnaire again on a subset of the individuals who have participated in the study. Hence we administered the same questions on 73 individuals after 1 month and here we found that in GPAQ Rho ranged from 0.40 to 0.59 [Table 3] for different activities with significant P values except for moderate intensity work where Rho was 0.21 and the associated P value was 0.08, while in IPAQ spearman correlation coefficient was 0.42 to 0.43 [Table 3] in different domains with significant P values except for moderate activity where Rho was 0.21 and the associated P value was 0.08. The ICC for GPAQ varied from 0.37 to 0.81, whereas for IPAQ it ranged from 0.39 to 0.68. In both GPAQ and IPAQ P values were not significant for moderate intensity work.

Table 3.

Reliability of GPAQ and IPAQ after 1 month

Concurrent validity

The observed concurrent activity validity coefficients suggested that both the question showed reasonable agreement [Table 4]. Spearman Rho was 0.90 or higher and was associated with highly significant P values. Spearman's Rho was 0.89-1.00 for different variables. The ICC ranged from 0.76 to 0.91 [Table 4].

Table 4.

Concurrent validity for GPAQ and IPAQ

Criterion validity

The criterion validity of self-reported GPAQ and IPAQ against pedometer is shown in Table 5. We plotted PA MET minutes and pedometer counts to see if there was any correlation between counts and minutes. Overall, there was significant agreement between self-reported PA and pedometer count, but the SD was higher, indicating that there was increased variability in those data. When we examined self-reported PA and pedometer count in men and women, we found higher agreement in females when compared with males [Table 6]. When we compared overall MET minutes and Pedometer counts, it was significant and Rho was 0.36 and 0.39 respectively with GPAQ and IPAQ. Intra-cluster correlation and kappa value ranged from 0.68 to 0.72 and 0.51 to 0.58 respectively.

Table 5.

Criterion validity of GPAQ and IPAQ

Table 6.

Criterion validation of GPAQ and IPAQ with pedometer counts

DISCUSSION

The increasing global problem of physical inactivity and the need for population surveillance and inter-country comparisons led to the development of IPAQ measure.[21,22] The IPAQ instrument underwent several stages of development and testing before being used for PA measurement.[8] Being tested and validated all over the world to use in different populations was one of the reason to use it for validation against GPAQ. GPAQ instrument is included in the Steps approach as a part of Non Communicable Diseases Surveillance by WHO in SEARO countries. GPAQ was designed to capture PA domain such as vigorous, moderate and transport-related activity during work and leisure-time separately.

In this study, we got identical responses to PA questions during the visits with less than a week gap. This could be because there was much less variability in day-to-day activity in the study population or due to a high recall of the respondents as well as interviewer. Therefore, we decided to use a greater gap between two visits to assess the reliability. However, this could result in biasing the result as activities could differ widely by season. The results of this study are in concordance with the other studies conducted with the aim to test the reliability and validity of the two instruments. One such study was carried out on 251 adults in Vietnam.[23] GPAQ and IPAQ were administered on 2 occasions. The participants wore pedometers and logged their PA for 7 consecutive days. Although the GPAQ reliability was poorer for transport (GPAQ r = 0.25, IPAQ r = 0.60) and for leisure (GPAQ r = 0.21, IPAQ r = 0.45), yet GPAQ estimates of total PA for participants with a stable work patterns were moderately correlated with IPAQ (r = 0.32). Trinh et al. did the study to assess the test-retest repeatability and criterion validity of the GPAQ in Vietnamese adults during the dry and wet seasons.[24] For validation purposes, participants wore an accelerometer during the 7 days before the first and last GPAQ assessments. Total GPAQ score and accelerometer data showed validity correlations of 0.34 and 0.20 in the dry and wet season, respectively.

The strengths of the current study include the involvement of the study sample from both rural and peri-urban areas, involving the use of pedometers as an objective way to measure PA and also the fact that it is one of the few studies from India, more so from the northern parts of the country, which looked into the feasibility of using GPAQ, in place of IPAQ in community based settings. There were certain limitations in our study. The use of the pedometers itself would have made the participants conscious and they could have started doing more PA compared with their normal routine. There can be a possibility that they may have forgotten to wear pedometers on certain occasions. Moreover, pedometers would not have been able to capture activity during swimming, cycling etc., As seen in the study, the identical responses to PA questions during the visits with less than a week gap could be because of a high recall of the respondents as well as interviewer. Furthermore, one of the important limitations of the study is that the analysis based on the gender of the participants could not been done.

CONCLUSIONS

Result of this study showed that GPAQ is equivalent to IPAQ for measuring PA. Based on findings of this study, it can be concluded that GPAQ can be used with confidence in place of IPAQ. Further research is recommended to find other socio-cultural differences in reliability and validity of GPAQ.