Development and Evaluation of a Questionnaire for Assessment of Determinants of Weight Disorders among Children and Adolescents: The Caspian-IV Study

Abstract

Background:

Little experience exists on valid and reliable tools for assessment of the determinants of underweight and overweight in children and adolescents living in the Middle-East and North Africa (MENA). This study aimed to develop a valid and wideranging questionnaire for assessment of these parameters in a nationwide sample of Iranian children and adolescents.

Methods:

This national study was conducted in 31 provinces in Iran. The first phase consisted of focus group discussion with 275 children and adolescents and their parents. After a qualitative content analysis, the initial items were extracted. In the next step, the face validity was assessed by expert panelists using the quantitative method of the Impact Score. To assess the content validity, the content validity rate (CVR) and the content validity index (CVI) were determined. The internal consistency was examined by Cronbach alpha, and its test-retest reliability was determined. The socio-demographic variables, perinatal factors, lifestyle factors, family history, knowledge and attitude were assessed. Dietary intakes were assessed by a validated 168-item semi-quantitative food frequency questionnaire. A validated questionnaire for quality of life was filled in anonymously.

Results:

A team of expert researchers conducted the data analysis of 576 interviews by using qualitative content analysis method. The analysis process began by determining the semantic units about the concepts studied. The initial questionnaire was developed in four domains by including Likert scale questions. In the face validity step, all questions of the primary questionnaire obtained a score of more than 1.5. In the phase of CVR assessment, 6 questions obtained a score of less than 0.62, and were omitted. The rest of questions were assessed for CVI, and got a score of more than 0.75. Cronbach's alpha coefficient of the whole questionnaire was 0.97, and the Pearson correlation coefficient of the test-retest phase was 0.94.

Conclusion:

The developed questionnaire is a valid and reliable tool for assessment of the determinants of weight disorders in a nationally representative sample of children and adolescents in the MENA.

INTRODUCTION

The pediatric age group has the unique feature of growth. Growth disorders in terms of underweight and overweight are common among children and adolescents, and have short-term and long-term health consequences.[1,2]

Commonly, nutritional deficiencies have been of concern for educational curriculum and health policies; whereas epidemiologic transition and lifestyle changes have increased various types of nutritional disorders at global level. Although in high-income countries, excess weight has become as the main nutritional disorder in various age groups, notably in children and adolescents, the low- and middle-income countries have a double burden of nutritional disorders. While still struggling with nutritional deficiencies, they are facing an escalating trend of overweight.[3–5]

The double burden of nutritional disorders among children and adolescents deserves multi-faceted policies with evidence-based local programs for each community. Thus, one of the first steps of developing effective, practical, and sustainable interventions would be the understanding of the complex nature of the determinants of nutritional disorders in each community. Most studies have been conducted either in high-income countries focusing on the determinants of childhood obesity or in very low-income countries with severe malnutrition. To the best of our knowledge, no previous nationwide study has developed a valid questionnaire to assess the determinants of underweight and overweight in children and adolescents living in the Middle-East and North Africa (MENA).

The general health status of Iranian children has improved considerably over the past decades; however, rapid lifestyle change notably sedentary lifestyle and tendency to a Western diet, is a threat for their health. National studies have documented dual burden of weight disorders in Iranian children and adolescents.[5,6]

This study aimed to develop a valid and wide-ranging questionnaire for assessment of these parameters in a nationally representative sample of children and adolescents in MENA.

METHODS

This questionnaire was developed for part of a national survey of school students’ high risk behaviors” of the school-based surveillance system entitled Childhood and Adolescence Surveillance and PreventIon of Adult Noncommunicable disease (CASPIAN [CASPIAN is the name of the world's largest lake, located in Northern, Iran]) Study.[7,8] The whole surveillance program includes various risk behaviors and risk factors, and because of the importance of dual weight disorders, as part of the fourth survey of the CASPIAN Study, a simultaneous nationwide study was conducted in 2011–2012 to assess the determinants of weight disorders. The current paper presents the steps considered for development and validation of a questionnaire for this part of the study.

The study was conducted according to the declaration of Helsinki (Seoul, 2008) and approved by the institutional review boards at national and provincial level. Participants were enrolled to the survey after complete explanation of the objectives and protocols. They were assured that their responses would remain anonymous and confidential, and no information that could reveal their identity will be used. Moreover, we informed potential participants that participation in the study is voluntary, that they may decline to participate in the study without any penalty, and that they have the right to withdraw from the study at any time. Written informed consent and oral assent were obtained from parents and students, respectively.

This qualitative study was conducted in urban and rural areas of the central counties of all provinces in Iran by the following steps:

First step

Focus group discussions were conducted separately for students and their parents. The concepts of “weight disorders, body image, underweight, obesity, lifestyle habits, dietary habits, physical activity pattern, behaviors, quality of life, family composition, socio-demographic factors, and genetic determinants” were defined and determined. For this purpose, students aged 6–18 years and their parents were selected by cluster sampling from each province. Sampling was based on objective and data gathering was continued until data saturation.

Before the interviews, the research objectives, reasons for recording the interviews, voluntary participation, confidentiality of information and the identity of persons interviewed and their right to participate or not to participate in the research, were all described. Focus groups were conducted in the form of moderated group discussions, with 5–10 participants, led by a facilitator. One or more observer took notes but did not have direct interaction in the session.

All interviewers had experience in conducting qualitative studies; in most counties they were selected from trained nurses. The place of conducting focus groups differed in each province; to facilitate the presence of students and parents, most interviews were performed in the schools at the end of the education time and or on holidays. Participants seated in a comfortable environment in a way that they could face one another. The facilitators followed a script, with a set of questions designed to produce feedbacks that would meet the purpose of the focus group. Typically each focus group started with a brief introduction by the facilitator, who explained the purpose of the focus group, and followed the script and questions by adding additional follow-up questions as needed. The interviews were formulated structurally. Each session lasted for 60–90 minutes, and was accomplished with a closing statement by thanking the participants. Then, the project team gathered the handwritten texts and listened to the tape-recorded voice on several occasions and typed them word by word. The handwritten texts were reviewed several times.

Documents of 576 interviews related to students and parents were sent to Tehran, where a team of expert researchers conducted the data analysis by using qualitative content analysis method.[9] The analysis process began by determining the semantic units associated with the understanding of participants about the concepts studied. Two researchers independently reviewed the interviews several times. The analysis process proceeded by coding the interviews line by line. Statements that were unrelated to the study were excluded. Then, codes containing similar meanings were put within categories. A code or category name was assigned for meaningful segment of text in a transcript to signify that precise segment. Then this process was continued until all data were segmented and the initial coding was completed.

To ensure the reliability of data, researchers retained the long-term involvement with the data; which increased the breadth and depth of information. In addition, the researchers tried to increase the accuracy and conformability of the research data by collecting real and acceptable data. Peer-reviewing and using their reforming views and also reviewing some of the extracted codes with some of the participants were considered for increasing the credibility of data.

After explaining the abovementioned concepts regarding the factors affecting the prevention, occurrence, and management of weight disorders, the initial categories were formed. The categories were re-reviewed, an extensive literature review was conducted regarding relevant issues, and existing questionnaires were verified, then similar categories were merged. Lastly, the initial questionnaire was developed in four domains by including 50 questions designed based on a five-point Likert scale. These domains included lifestyle habits, awareness, and attitudes toward thinness and excess weight.

Second step

In this stage, the validity and reliability of the questionnaire were assessed. As selection of 100 individuals is adequate for assessment of the accuracy of validity and reliability of a questionnaire,[10,11] the preliminary questionnaire was filled in by 100 students and an equal number of parents. To assess the content validity, the content validity rate (CVR) and the content validity index (CVI) were determined. Afterwards, the questionnaire's internal consistency was examined by Cronbach alpha, and its test-retest reliability was determined. SPSS software package (version 16; SPSS Inc., Chicago, IL, USA) was used for data analyses.

Face validity was determined by asking the points of views of experts and a sample of the target group. Ten faculty members of various specialties related to the fields of pediatrics and health assessed the questionnaire for the face validity, reasonableness, appropriateness, attractiveness, and logical sequence of items. In addition, 20 persons randomly selected from the target group presented their views about fluency and understandability of the items.

Then, the quantitative method of impact score was used to reduce and remove inappropriate items, and to determine the importance of each item. The revised questionnaire was given to 100 students of different academic levels and one of their parents, and then the score of each question was calculated separately. The five-point Likert scale answers consisted of: Very important (5 scores), important (4 scores), averagely important (3 scores), slightly important (2 scores), and not important (1 score). Questions that received a score of more than 1.5 were retained for subsequent analyses.[12] At this stage, all questions obtained high scores and there was no eliminated question.

To examine the CVR, the questionnaire was given to 10 experts in the specialties related to the field of the study; the answers were designed based on a three-point Likert scale consisting of: Necessary, helpful but not necessary, and not necessary. Then the questionnaire's CVR was assessed; according to the Lawsche table if the item score was over 0.62, the item was considered as an appropriate and necessary one.[12] Regarding the obtained scores at this stage, those comments and views of the respondents and the re-thoughts on the items with lower scores, those that seemed unable to measure the desired concept, or those that had a little connection with the issue were excluded.

To distinguish between CVI of the means and Waltz and Bausell's CVI, the views of 20 faculty members of related fields were used. The indexes of “relevance”, “clarity”, and “simplicity” examined the questions of the questionnaire based on four-point scale.[13] Moreover, the experts were asked to respond to two questions: (1) the viewpoints they believed that should be imposed and (2) suggestions for the items that should be entered into the questionnaire. A separate CVI was calculated for each item. Items with the score of over 0.75 were retained as appropriate ones.[13]

To assess the questionnaire's reliability, its internal consistency and homogeneity, the internal consistency reliability coefficient was calculated by Cronbach's alpha. Usually alpha should be equal to or greater than 0.8 (alpha ≥ 0.8); however, alpha = 0.7 is also acceptable.[14] The final questionnaire was given to 100 students from different academic levels and one of their parents.

The test–retest reliability was used to check the stability. Therefore, 40 students from different academic levels and one of their parents completed the questionnaire twice with an interval of 2 weeks. The One-sample Kolmogorov–Smirnov test represented a normal distribution of the obtained scores; therefore, Pearson correlation coefficient was assessed between the two completed questionnaires.

Given the complexity of factors associated with underweight and overweight, other aspects of possible determinants were asked by using validated questionnaires used in our previous studies. The socio-demographic variables, perinatal factors, type of milk and weaning food in infancy, changes in family lifestyle habits in the past year, family history of chronic diseases, dietary habits, and physical activity pattern were assessed by the questionnaires of the surveillance program.[7,8]

Usual dietary intakes of children and adolescents were assessed using a validated 168-item semi-quantitative food frequency questionnaire (FFQ). The FFQ consisted of a list of foods with a standard serving sizes commonly consumed by Iranians. Interviewers asked participants and their parents to report the frequency of consumption of children and adolescents of a given serving of each food item during the previous year on a daily (e.g., bread), weekly (e.g., rice, meat), or monthly (e.g., fish) basis. The reported frequency for each food item was then converted to a daily intake. Total energy intake was calculated by summing up energy intakes from all foods.[15]

Children and adolescents were asked to fill in the anonymous quality of life questionnaire, which was prepared for two age groups of 6–13, and 13–18 years.[16]

RESULTS

In the stage of assessing the questionnaire's CVR, 6 items were removed and 44 items entered the second stage for the measurement of the questionnaire's CVI. There was no eliminated question in the CVI assessment, and all the questions had a score above 0.75.

The Cronbach's alpha and intra class correlation coefficient of main domains studied are presented in Table 1. The Cronbach's alpha coefficient for assessment of the questionnaire's reliability, was 0.97 for the total means and in the range of 0.92–0.96 for the four domains studied (physical activity: 0.92; attitude: 0.94; awareness: 0.95; and diet: 0.96).

Table 1.

Cronbach's alpha and intra class correlation coefficient of main domains

The test–retest reliability was used to check the stability. The Pearson correlation coefficient (r = 0.94) indicated the good stability of the questionnaire.

DISCUSSION

This study was unique in studying a nationally representative sample of children and parents living in the MENA region. It provided a valid and reliable questionnaire to assess the determinants of growth disorders, in terms of underweight and overweight, among children and adolescents. A valid and reliable tool is necessary for further interpretations of the study findings, and their implication in future health policies. In this nationwide study, the content validity was evaluated by the help of a team of expert specialists. In general, the assessment of reliability and validity showed that the whole questionnaire had an acceptable validity and reliability. The reproducibility of most of the items in this questionnaire was similar to the ones previously designed in other communities. For instance, the questions about healthy nutrition, physical activity, and weight change provided in the current study are consistent with some previous questionnaires designed in Western countries.[17,18] The dimensions of our designed questionnaire were in line with the dimensions of the questionnaire about nutritional monitoring of school-aged children in the study of Hoelscher et al.[18] Likewise, our results are in agreement with the study of Story et al., who designed a questionnaire and tested its validity and reliability for assessment of weight gain and weight loss, attitudes toward body size, diet, and physical activity among elementary school children.[19]

In 2009, Leung et al. designed the SCOFF questionnaire and tested its validity and reliability in order to screen eating disorders in school-aged children in Hong Kong. This questionnaire included four dimensions of nutritional restraint, body image and shape concern, overweight concern, and eating concern that are nearly consistent with two dimensions of our study's questionnaire. The content validity of this questionnaire was achieved, same as ours, through the panel of experts and the reliability of the dimensions was calculated, respectively, as follows: 0.86 for nutritional restraint, 0.86 for appearance and shape concern, 0.83 for overweight concern, and 0.9 for eating concern, which accords with the results of our questionnaire. The internal reliability of this questionnaire was acceptable and consistent with the results of our study as well.[20] Our findings are also consistent with the questionnaire developed by Swift et al. They designed and tested its validity and reliability for evaluating the knowledge of adolescents regarding the health risks of obesity. Their questionnaire had an alpha coefficient of 0.70.[21] In 2009, Neuhouser et al. validated a 19-item questionnaire about nutritional behavior in school students. The questionnaire's reliability was calculated, through the test-retest; it was 0.85 for fruit and vegetable consumption, the corresponding figure was 0.74 for the consumption of snacks/sweets, and 0.72 for carbonated beverages and junk food, respectively.[22]

Our findings are confirmatory evidence on the usefulness of designing a questionnaire for the study of nutritional knowledge both in children and adolescents and in their parents. Swift et al. and Neuhouser et al. showed that their designed questionnaires could indicate the relationship between the awareness of health effects of weight disorders, and weight control behaviors.[21,22]

In 2010, Branscum et al. evaluated the validity and reliability of a modified check list of nutrition behavior in children. Although they found that this tool has been a proper means for assessment of dietary behavior among children, but the alpha coefficient was not high and ranged between 0.43 and 0.67 for the dimensions of this questionnaire.[23] The lower coefficient of this study than in ours might be because of the fewer number of questions and dimensions of this questionnaire than ours.

The questionnaire validated in the current study has some distinction points with the previous ones, it comprises both underweight and overweight, in addition to the usual questions on knowledge, attitude, and behavior related to nutritional and physical activity habits, it includes various possible determinants of weight disorders from perinatal and infancy period, as well as quality of life and different socio-economic factors. This questionnaire can serve as a comprehensive and valid tool for a complete evaluation of weight disorders. Moreover, it is appropriate for the socio-cultural issues of families living in developing countries. Nonetheless, most previous studies have been conducted in industrialized countries, and the questionnaires included only some aspects of factors related to weight status with focus on overweight.

Study limitations and strengths

The main limitation of the questionnaire developed in this study is the high number of its questions; however, most participants and experts evaluating the questionnaire found it appropriate. The main strength of this study is that the questionnaire that was developed has been designed based on several focus groups with the students and parents in various provinces of the country. Therefore, this design could represent the comprehensiveness of the items extracted from the interviews. The other strength is its novelty in comprising different possible determinants of both underweight and overweight in a nationally representative sample of children and adolescents living in MENA.

CONCLUSION

The results of this study showed that this questionnaire is a valid and reliable tool for screening weight disorders in children and adolescents. It might be generalized to other pediatric populations in the MENA. Its findings can be used by health policy makers and might be useful at individual and public health levels.