Psychometric properties of questionnaires to measure social ecological influences in Vietnamese children

Abstract

Physical activity data in primary school-aged children are limited in Vietnam. Although tools to measure social ecological influences on physical activity are validated in English, they are not available in Vietnamese. Due to cultural and contextual differences, their psychometric properties need to be tested. Five scales were translated into Vietnamese and evaluated for internal consistency and test re-test reliability, including self-efficacy, perceived social influences, and beliefs self-administered by students, and parental support for physical activity and parental perceived safety of the neighbourhood, self-administered by parents. Compared to the original scales, two items from the parental perceived neighbourhood safety were removed due to the cultural context. Another item of the self-efficacy scale was also removed as it correlated poorly with the other items in the scale at both administrations. The adjusted scales were found to be reliable and appropriate for use among students and parents to measure social ecological influences on physical activity in the Vietnamese context.

Introduction

Physical activity (PA) is an important lifestyle factor that is associated with multiple health-related conditions.^1,2 However, a majority of children in many countries do not meet the PA recommendation of engaging in moderate-vigorous PA (MVPA) for at least 60 min/day.3, 4, 5 In many low and middle income countries (LMIC) including Vietnam where urbanization and mechanization are resulting in a more sedentary lifestyle, population-based PA data objectively measured in primary school-aged children are limited. Our recent study in urban areas of Ho Chi Minh City (HCMC), Vietnam showed that only about 18% of children met the PA recommendation and 52.7% were overweight/obese⁶ which clearly emphasized a need for interventions to improve PA. However, information about correlates of PA among primary-school aged children needed for designing and implementing effective interventions was not available in Vietnam.

Although many instruments to measure correlates of PA among children are available, their psychometric properties have not been examined among Vietnamese children. As the target population are young children, testing the instruments is necessary to provide information to improve the questions accounting for the children's cognitive development. There are also differences in Vietnamese culture and context. For example, the most common means of transportation are motorbikes whereas cars are not popular in Vietnam. The local streets are usually used not only for transportation but also as playgrounds for children and as retail sites. Therefore, it was important to ensure that psychometric properties of instruments are tested before using them in different cultural contexts.

Five scales reflecting intrapersonal (self-efficacy, beliefs, and perceived social influences), interpersonal (parental support for PA), and environmental levels (parental perception of neighbourhood safety) were selected due to their important roles and potential effects on children's PA. The first three scales were originally developed by⁷ in English but were commonly used among fifth-grade students in South Carolina,⁸ USA, and third-to-fifth-grade students in Virginia,⁹ USA, and adolescents in Kuantan, Malaysia¹⁰ and Singapore.¹¹ The parental support for PA and parental perception of neighbourhood safety, which are designed for parents/caregivers to complete, were developed respectively by Trost et al. 2000¹² and Carver et al. 2008¹³, and used in other studies in the U.S¹⁴ and Australia.¹⁵

This study specifically aims to (1) develop the Vietnamese version of the above questionnaires to measure socio-ecological influences on children's PA, and (2) test psychometric characteristics of these Vietnamese versions.

Methods

Description of the instruments

The instruments to be tested included two questionnaires: one to be self-administered by students under supervision in the classroom and the other to be self-administered by their parents at home. The student questionnaire included three scales: self-efficacy, perceived social influences, and beliefs. The parent questionnaire included two scales: parental support for PA and parental perceived safety of the neighbourhood.

The first three scales were developed and validated with fifth-grade students in the U.S. by Saunders et al. 1997.⁷ The self-efficacy scale has 17 items asking (i) whether students have support from parents/other adults or friends for PA; (ii) whether students are confident to overcome barriers to PA; and (iii) whether students have positive alternatives to PA. The perceived social influences scale has eight items to assess influences of family and friends on PA. The beliefs scale has 16 items asking students about consequences of being physically active. Responses for each item in these three scales are recorded as yes/no.

The parental support for PA scale which was developed by¹² has five items asking during a typical week, how often would a mother/female or father/male adult in the household (i) “encourage your child to do physical activity or play sports”; (ii) “play outside with your child or do physical activity or sports with your child”; (iii) “ride or provide transportation to a place your child can do physical activity or play sports”; (iv) “watch your child participate in sport, physical activities, or outdoor games”; and (v) “tell your child that sports or physical activity is good for their health”. Responses for these questions are “never”, “<1 times/week”, “1–2 times/week”, “3–4 times/week”, “5–6 times/week”, or “daily”.

The parental perceived safety of the neighbourhood scale which was developed by Carver et al. 2008¹³ has 14 items asking about parents’ perception of road safety, incivilities, and personal safety of a child. Each question has a response of “strongly agree”, “agree”, “disagree”, and “strongly disagree”.

Development of the Vietnamese versions of the scales

Translation

The practice suggested by Beaton et al. (2000) was modified and applied.¹⁶ The validated scales in English were translated into Vietnamese by the first author and then back-translated into English by an independent Vietnamese researcher fluent in both English and Vietnamese. The back-translated version was then compared with the original instrument by a native English speaker. Any disagreement between the back-translated and original versions (for example, the original version of “to get me the equipment” was translated into “to buy me the equipment”) were discussed, negotiated and adjusted accordingly. The Vietnamese versions of the scales were then reviewed by two Vietnamese experts to ensure their clarity and appropriateness to local culture and context.

Cognitive interview

After translating the scales into Vietnamese, the student and parent/caregiver questionnaires were evaluated on how participants understood, processed and responded to the scale items using a method of cognitive interviewing.¹⁷ This is important to ensure validity, especially among young children. In this study, parents and students were interviewed regarding both the format and content of the questionnaires. Discussion around the format included font size, type, and design of the questionnaires for readability. The content discussion included checking the understanding of each question, exploring responses and verifying answers. General probes and probes for specific questions were used to test issues related to wording, technical terms, reference periods, and vagueness.

Five 5^th-grade students (three girls and two boys) and five parents (three mothers and two fathers) participated in the cognitive interviewing portion of the study. Prior to the interviews, parents provided written consents and students were asked for their assent. To be eligible, participants needed to be literate and able to complete the self-administered questionnaire. All interviews were conducted on-site at school.

As a result of the interviews, changes in format and wording were made including increasing space between sentences; underlining or italicizing words for emphasis; replacing a matrix format with a simple text format; and using words “less than” instead of the symbol “<”. Items 5 and 13 were removed from the parental perceived safety scale due to the environmental context in HCMC in which fifth-grade students almost never take the bus to school and there are few traffic slowing devices in neighbourhoods.

Evaluation of psychometrics

After the cognitive interviewing phase, the questionnaires were administered to parents and students from a single school to determine the psychometric properties of each scale.

Participants and setting

Two urban primary schools in HCMC were purposively selected for the study. Students in four 5th-grade classes (n = 129 students) from one school were invited to complete the student survey on two occasions. Written consent forms were given to students to take home for their parents/caregivers to sign and return to school if they agreed for their children to participate. As a test-retest on parents of 5^th grade students in this school was unable to be conducted, the parents of students from two 4^th-grade classes (n = 80 parents) in a neighbouring school from the same school district were invited to participate. Ethics approval from the Queensland University of Technology Human Research Ethics Committee was obtained (1500000549). Written approvals from the HCMC Department of Education and the schools were also received.

Data collection

The student questionnaire was administered during class time on a Monday morning at the beginning of the school day. A research assistant, provided instructions and supervised students while students completed the questionnaire. For parents/caregivers who participated in a test-retest, the questionnaire was placed in an envelope and given to students to take home for them to complete. Students were asked to return the completed questionnaire to school the following day. The student and parent questionnaires were re-administered using the same protocol one week later to evaluate test-retest reliability. Test-retest was assessed over a period of one week to match previous studies and to ensure that there was no change in the underlying construct being measured. If repeat tests were made over longer periods then we would be evaluating stability of the underlying construct rather than reliability. If repeat tests were made over shorter periods (within days) then assessments may have been positively biased by knowledge of previous responses.

Data analysis

SAS Version 9.4 and IBM SPSS Statistics Version 22 were used for the analyses. For the self-efficacy, social influences, and beliefs scales, a response to each item was assigned a score (yes = 1, no = 0) except that scores were reversed for items 2, 6, 8, 10, and 11 of the beliefs scale. For the parental support for PA scale, a score of 1–6 was assigned respectively to a response of “Never” to “Daily”. A score of 1–4 was assigned respectively to a response of “Strongly agree” to “Strongly disagree” for the parental perceived neighbourhood safety scale; except for items 5, 9, 10, and 11, scores were reversed. Internal consistency reliability was assessed using Cronbach's alpha.

Total scores were calculated for each scale by summing scores of single items. These scores were used to compute Intra-class Correlation Coefficients (ICC) between two administrations (single measures coefficients were reported from two-way mixed effects model with absolute agreement).

Results

There were 129 students and 80 parents invited to participate in the study. A total of 99 students (54 boys and 45 girls) and 61 parents completed the first administration of the questionnaires, resulting in a response rate of 76.7% for students and 76.3% for parents. However, 96 students (74.4%) and only 41 parents (51.3%) completed the repeat administration of questionnaires one week later. At pre-test, students' average age was 10.1 (SD = 0.3) years. Parents’ average age was 38.6 years (SD = 4.8) with 40% having an education level of high school/above at pre-test and 38.2 years (SD = 6.2) with 63.3% having an education of high school/above at post-test.

Internal consistency reliability

Cronbach's alphas for the scales are presented in Table 1. All scales had moderate internal consistency reliability (alphas from 0.61 to 0.76).¹⁸ Due to low item to total score correlations (<0.06), item 17 in the self-efficacy scale that asked whether students think they “can be physically active at least three times a week for the next 2 weeks” was removed from the final questionnaire.

Table 1.

Internal consistency reliability for five scales.

Scales	items	α
Self-efficacy	16	0.63
Perceived Social Influences	8	0.62
Beliefs	16	0.67
Parental Support for physical activity	5	0.76
Parental Perceived Neighbourhood Safety	12	0.74

Test-retest reliability

Test-retest reliability for the five scales is presented in Table 2. Self-efficacy, beliefs, and parental support showed good reliability, with ICCs ranging from 0.71 to 0.77. Reliability was fair for social influences (ICC = 0.57) and parental perceived neighbourhood safety (ICC = 0.56).¹⁹

Table 2.

Test-retest Intra-class correlation coefficients (ICC) for each scale.

Scale	n	ICC (95% CI)
Self-efficacy	96	0.71 (0.59, 0.79)
Perceived Social Influences	93	0.57 (0.42, 0.70)
Beliefs	87	0.77 (0.67, 0.85)
Parental Support for physical activity	39	0.73 (0.55, 0.85)
Parental Perceived Neighbourhood Safety	41	0.56 (0.29, 0.74)

Discussion

This study was designed to develop and test psychometric characteristics of the Vietnamese version of questionnaires to measure socio-ecological influences on children's PA. Compared to the original scales, two items from the Parental Perceived Neighbourhood Safety scale were removed due to the cultural context. The first item asked about “traffic slowing devices (e.g., speed humps) in our local streets”. These devices are extremely rare in the neighbourhoods in HCMC, and probably throughout Vietnam because the local streets are too narrow for cars to be able to enter, allowing only motorbikes and bicycles. These streets are also usually filled with families undertaking activities of daily life as well as shopkeepers. Therefore, the traffic is usually very slow even though there are no speed regulations or traffic slowing devices.

The other item asked about whether it was “safe walking home from a bus stop or train stop at night”. In HCMC, trains are not currently used to travel inside the city. Although buses can be used, they are not popular even among adults. Buses cannot move through local streets and bus stops are mostly located at a distance far from houses. There are also concerns about the safety of children using buses resulting in fifth-grade students in HCMC rarely using them.

Another item of the Self-efficacy scale was also removed as it correlated poorly with the other items in the scale at both administrations. Another reason to remove this item was due to the high level of difficulty for children to answer (that is have you been physically active at least three times a week for the next two weeks). This may have prevented children from responding accurately.

In addition to the content of items in the scales, it was sometimes challenging for participants with low levels of education to answer self-administered questions placed in a matrix format although this format looks neat and saves space. A simple text format with the responses repeated for each question was a better option and therefore was used. Signs or symbols such as “<” can be misunderstood, particularly for those with a lower level of education, and were replaced by texts.

Findings from this study also showed that the psychometric characteristics of the Vietnamese versions were good with ICC values for all scales being fair to good. Cronbach's alphas for the three scales (self-efficacy, perceived social influences, and beliefs) were below the general cut-off of 0.7 but comparable to those conducted among third-to fifth-grade African American girls (0.53–0.83)⁹ and fifth-grade students in rural South Carolina (0.54–0.75)⁸. However, given the cognitive development of 5^th grade children, we consider these scales reliable and appropriate for use among 5^th grade Vietnamese children.

Previous studies using the self-efficacy, perceived social influences and beliefs scales had response options in either a 5-point or a 3-point Likert scale; however, these were usually conducted among those in middle schools.^10,11,20,21 Our study, along with other studies among elementary students, followed the original study using a yes/no option.7, 8, 9^,22 The loss of variability is acceptable to account for the children's cognitive development.²³

This study has some limitations. First, the response rate at the second administration was not high for parents (51.3%). It was challenging to have the parents complete the questionnaire twice as many of them were unable to commit the time required. As parents with complete data at pre- and post-test had a higher level of education than those with only pretest data (63.3% vs. 40% with high school degree/above), we cannot rule out the possibility that more educated parents completing both assessments would understand the questions better than those with a lower level of education, and that the estimates of test-retest reliability may be positively biased. However, the cognitive interviews were conducted on parents with a very low education level and we found little evidence that parents with low education did not understand the items. Second, because the school was not willing to send questionnaires to parents a second time, the parents of fourth-grade students from a different school completed the test-retest evaluation. However, because these parents were recruited from a neighbouring school in the same school district, it is expected that the characteristics of these parents would not be different from those of fifth-grade students in the other school. Finally, the study was conducted in two schools in a single school district and the results may not be generalizable to all children and parents in Vietnam. To conclude, the adjusted scales were found to be reliable and appropriate for use among 5^th grade students and parents to measure social-ecological influences on PA in the Vietnamese context.

Conflict of interest

The authors declare no competing interests.

Submission statement

This manuscript is not currently submitted elsewhere. None of the manuscript's contents have been previously published in any journal. All authors have read and approved the submitted manuscript.

Each authors’ contributions

All authors significantly contributed to the manuscript. QGT, SGT, DG, DVD, HTMT, KGT, LW designed the study. QGT, KGT, DVD, HTMT collected data. QGT drafted the manuscript. QGT, SGT, DVD, KGT analysed the data. QGT, SGT, DG, DVD, HTMT, KGT, LW contributed to interpretation of data. SGT, DG, DVD, HTMT, KGT, LW critically reviewed the manuscript for important intellectual content.