Development and psychometric validation of a scoring questionnaire to assess healthy lifestyles among adolescents in Catalonia

Abstract

Background

Lifestyle is intimately related to health. A questionnaire that specifically scores the healthiness of lifestyle of Catalan adolescents is needed. The objective of this study was to develop and validate a scoring questionnaire called VISA-TEEN to assess the healthy lifestyle of young Catalans that can be answered quickly and user-friendly.

Methods

A lifestyle questionnaire was developed based on the analysis of contributions from two focus groups, one with adolescents and the other with people who work with them (teachers and doctors). A panel of experts validated the content of items that were ultimately selected for the VISA-TEEN questionnaire. Three hundred ninety-six adolescents (215 boys and 181 girls, age = 13–19 years) completed the VISA-TEEN. Internal consistency was assessed using Cronbach's alpha (α) reliability coefficient. Test-retest reliability, using an intraclass correlation coefficient (ICC), was calculated based on scores attained two weeks apart. Construct validity was assessed by the extraction of components with an exploratory factor analysis. The relationship between the scores was measured using the health-related quality of life (HRQoL) KIDSCREEN-10 Index (the relationship was assessed by calculating Pearson’s r correlation coefficient). The association of scores in the VISA-TEEN for self-rated health (SRH) was also examined by executing an analysis of variance (ANOVA) between the different categories of this variable. We also calculated the index of fit for factor scales (IFFS) for each component, as well as the discriminatory power of the instrument using Ferguson’s δ (delta) coefficient.

Results

The VISA-TEEN questionnaire showed acceptable reliability (α = 0.66, α_est = 0.77) and a very good test-retest agreement (ICC = 0.860). It could be broken down into the following five components, all with an acceptable or very good IFFS (0.7–0.96): diet, substance abuse, physical activity, Rational Use of Technological Leisure (RUTL), and hygiene. Scores on the VISA-TEEN showed significant correlation with the KIDSCREEN index (r = 0.21, p < 0.001) and were associated with SRH (p < 0.001). The discriminatory power was found to be δ = 0.97.

Conclusions

The VISA-TEEN questionnaire developed to study the lifestyle of Catalan adolescents is a valid instrument to apply in this population as it is shown in the present psychometric tests to understand the role of lifestyle in the health of teenagers or to test the efficacy of health campaigns intended to improve teenagers' lifestyle.

Electronic supplementary material

The online version of this article (doi:10.1186/s12889-016-2778-6) contains supplementary material, which is available to authorized users.

Background

There is a clear relationship between the way people live and their health, which is why we speak of healthy and unhealthy lifestyles. The World Health Organization (WHO) defines lifestyle as the way of living based on identifiable patterns or behaviours, which are determined by the interaction between individual and personal characteristics, social relations, and socioeconomic and environmental living conditions [1]. Regarding adolescents, the WHO survey of Health Behaviour in School-aged Children (HBSC) is an instrument of analysis of lifestyle behaviours related to nutrition, physical activity, relaxation, addiction, injuries, hygiene, and sexuality [2].

There is evidence concerning to what extent and in what ways the behaviours assessed in the HBSC influence adolescent health in the areas of nutrition [3–11], physical activity [12–18], relaxation [19–26], addictions [27–39], and personal hygiene [40–43]. That is why creating questionnaires that assess lifestyle at this age is crucial to study all the dimensions of public Health in this population.

There are currently several tools to assess and rate the healthiness of lifestyle, including FANTASTIC [44–46], Health Promoting Lifestyle Profile II (HPLP II) [47], and Personal Lifestyle Questionnaire (PLQ) [48]. All of these tools are validated but either are not specific for adolescents or are antiquated and do not include certain lifestyles of today's teens (for example those habits related to the use of new technologies and online contacts through social networking). Moreover, some were developed on the basis of different Spanish cultures [49], or are long and require an excessive time burden on the respondent. Other studies use questionnaires prepared ad hoc, evaluating only certain dimensions related to lifestyle, most of them related to nutrition and physical activity issues [50, 51]. In all cases, it was concluded that lifestyle significantly affects the health of adolescents [52–55].

The use of questionnaires as an evaluation tool in the field of health is widespread but it is necessary to have adequate criteria in order to obtain good quality of the information assessed. The Scientific Advisory Committee (SAC) of the Medical Outcomes Trust proposed a several useful criteria [56] as the basis to develop standardised assessment tools for measurements from patient reports, which are known as Patient-Reported Outcome Measures (PROMs). These criteria include the assessment of the psychometric qualities of the instrument, the need for participation of the target population in the conceptualisation phase, and taking into account the effects of the administration, both for those who administer and those who respond to the questionnaire. In 2008, Valderas et al. publish a study was aimed to develop a tool for the standardized assessment of patient-reported outcomes (PROs) to assist the choice of instruments [57].

Because lifestyle is largely conditioned by the environment, an instrument to assess lifestyle in adolescents in Catalonia (Spain) should be developed taking into account the characteristics of this age group and the cultural characteristics of the society in which they reside.

The objective of this study was to develop and validate a scoring questionnaire to assess the healthy lifestyle of Catalan adolescents (VISA-TEEN) that is also user-friendly and can be answered quickly. The questionnaire was called VISA-TEEN, an acronym from Spanish language (VIda SAludable = Healthy Life) and Teen, as the participants were adolescents.

Methods

Ethics

Ethics approval was obtained from the Research Ethics Committee (Faculty of Psychology, Education and Sport Sciences, University Ramon Llull). The study was conducted in accordance with the tenets of the Declaration of Helsinki. This study was a self-administrated questionnaire, and at the end, the decision to answer it was from the own students. The school accepted to participate in the project, and the Head of the institution signed the consent. The questionnaire was anonymous, and we guaranteed that the data and the results were going to be totally confidential, and they would only be used globally (the sample of all the schools), not to evaluate particularly each student. The confidentiality was also maintained during data analysis by delinking questionnaire data from any personal identification information. For those under 16, the principal's written agreement was first obtained. The regional regulations state that Catalonia school boards and principals should inform parents and collect their consent regarding any extracurricular activity. The research ethics committee also confirmed that the Head’s signature covered the permission to analyse the data of each class group.

Development of VISA-TEEN

The project was developed in two phases. The questionnaire was developed in the first phase and validated in the second phase. Previously, a review of the definitions of "lifestyle" and its relation to health was conducted. The focus of the search was on those investigations that limited the study to the field of adolescence. For this purpose, the PubMed, PsycINFO, SPORTDiscus, and SciELO databases were consulted, and two focus groups were subsequently organised. The literature review yielded information on the types of questions from existing questionnaires. Evidence of the influence of several indicators on the health of adolescents was also obtained from literature, as well as recommendations provided by various institutions regarding this.

The participation of the population under study was effective based on the results of two focus groups. One group consisted of eight adolescents from four schools in the city of Barcelona. The criteria to select the adolescent participants were based to obtain the highest representation. The other group was made up of five professionals in the following fields: adolescent medicine, adolescent eating behaviour disorders, psychology, information technology and communication, and a teacher specialising in high-risk adolescents. In both groups, the discussion began with the moderator prompting them to describe the day of an adolescent from rising until going to sleep. The two sessions were recorded and transcribed. Contents analysis was the one we applied. Through a process of segmentation and codification, we got a list of indicators. Using TextSTAT^© 2.8.g software by Matthias Hüning, a list of frequently used words was extracted. These words were then analysed in context using ATLAS.ti^© Version 6.0 software developed by ATLAS.ti^© Scientific Software Development GmbH, and a first list of lifestyle indicators for adolescents was developed. This first list was submitted for the judgement of a panel of six experts in health, anthropology, and education and health care. The experts were asked to point out which of the indicators could be used to value the adolescents' lifestyle. A second list of the most common indicators obtained according to those experts' opinions was made, and later the process was repeated with this second list to get a final list of indicators that should be included in the questionnaire.

With this information, we proceeded to the development of the questions. Mixed responses were chosen (some ranking, others numerical, and others closed-response multiple choice). Some questions could contain more than one item. A score rating between 0 and 3 was assigned to each item depending on the response and based on the influence on health as evidenced in the literature review as follows: 0 points for the least healthy response, not suited to the recommendations; and 3 points for the healthiest response, exactly suited to the recommendations. If the answer was not at either of these two extremes, it was scored with 2 points if it promoted health or 1 point if it could be harmful. The questionnaire was completed with a number of socio-demographic variables (month and year of birth, country of birth of the adolescent and their parents, sex, height, and weight) that were subsequently used to analyse possible relationships with these lifestyle factors.

The instrument was developed with the aim of being appealing to the target population and with a design such that it could be answered quickly. The layout was designed as a DIN-A5 booklet format with colour front and back covers and a white interior. One or two questions were included on each page, and it was illustrated with drawings to make the instrument more user-friendly.

The questionnaire was distributed to 67 adolescents at a secondary school in Barcelona. They were asked to give each item a quantitative and qualitative assessment of comprehensibility. Using the information collected, we proceeded to reformulate some statements, and questions were reordered following the proposals made by some of the adolescents. Likewise, the number of possible responses to the closed-response questions was reduced. The resulting questionnaire consisted of 11 questions drawn from 15 scoring items. This was the questionnaire that was used in the second phase.

Validation of the VISA-TEEN questionnaire

The second phase was the psychometric validation of the questionnaire. It was given to a sample of 419 adolescents on two occasions, 15 days apart. The participants were from five institutions in Catalonia, three of which were public and two of which were privately owned. Three institutions were classified as urban, one was suburban, and one was rural. Participating students were 13 to 19 years of age. Of the first set of questionnaires, 396 were returned, and 253 were returned from the second set. Questionnaires were identified with a code to ensure anonymity. On the first occasion, the KIDSCREEN-10 questionnaire [58], which assesses HRQoL in adolescents, and the SRH [59, 60], which provides information on perceived health status, were also administered.

Reliability: internal consistency and temporal stability

To assess reliability, temporal stability was analysed by calculating the ICC between scores from the first and second occasions, internal consistency was analysed using Cronbach's α coefficient and stratified α (indicated when the scales have more than one dimension) [61–63], and individual item analysis was analysed by calculating the corrected item-total correlations and α, if the item was removed. Reference values for assessing the ICC were proposed by Domenech [64], where values above 0.41 were considered moderately good, and values above 0.75 were very good. To assess internal consistency, we used reference values for α proposed by Morales [65], who considered values greater than 0.6 as acceptable in descriptive population studies.

Content validity, criterion, and construct

Content validity was confirmed from the analysis of the indicators executed by the panel of experts in the development phase of the questionnaire. Construct validity was assessed by testing the factorial structure with exploratory factor analysis (EFA). The principal components method was used, and the rotated matrix was extracted with varimax orthogonal rotation. In addition, once the components were obtained, the IFFS was calculated for each component. This index can replace the coefficient α of each factor when the factors are composed of a small number of items [66]. Values greater than 0.60 are considered acceptable IFFS, and values greater than 0.80 are considered “very good” [67].

The correlation between the score on the questionnaire and that obtained in the KIDSCREEN-10 was studied with Pearson’s linear correlation coefficient. The association of the scores with the SRH was assessed in order to analyse the differences between the various categories of this indicator. Higher scores were expected in those adolescents who showed better health. The analysis of these differences was performed using a one-way ANOVA with post hoc contrasts. Scores from each component (factor scores) were calculated as the mean scores of the elements that composed it. The association of scores on the various components of the questionnaire with sex, age, and purchasing power, measured with the Family Affluence Scale (FAS), was also analysed. To avoid potential interactions, these associations were assessed based on a multiple linear regression analysis. Additionally, the provenance of the adolescent was introduced into the equation to assess potential confounding effects. The accepted level of significance for all tests was 0.05.

Criterion validity had to be reaffirmed from content validity and construct [68] due to the lack of an error-free “Gold Standard” that assesses the healthy lifestyle of adolescents. The discriminatory power of the questionnaire was evaluated by calculating Ferguson’s δ coefficient modified for the total score of the questionnaire.

The results of changing the scores or eliminating any of the items were assessed, and the definitive version of the instrument was obtained.

The calculation of the scores and the analyses of different parameters were performed using IBM SPSS Statistics 20.0 software.

Results

Characteristics of participants

Table 1 summarises the characteristics of the participants (sample size, descriptions of age and sex) in each of the phases of development and validation of the questionnaire.

Table 1.

Age and sex of participants in the different phases of the study

	n	Age: (DE)	Boys (%)/Girls (%)
Processing step (focus groups)	13	—	—
Comprehensibility analysis	67	14.31 (0.97)	55.0 %/45.0 %
Validation phase	396	15.38 (1.58)	54.4 %/45.6 %

Development of VISA-TEEN questionnaire

From the literature review, it was found that lifestyle, when framed in the field of health and youth, identifies with eating habits, physical activity, addiction, relaxation, hygiene, sexuality, and injury.

Textual analysis of the transcripts of the focus group extracted a list of 37 concepts, of which the most prevalent were "parents" (frequency of appearance (f_ap) = 48), followed by "friends" (f_ap = 33), "Messenger/Facebook/mobile/talk (on-line)" (f_ap = 26), and "sport" (f_ap = 22). For the concepts of "play/game (computer, consoles)", "eat", "sleep", and "smoking", the frequencies were 13, 12, 9, and 5, respectively.

The analysis of the 37 concepts in context resulted in a total of 31 identifiable codes that, once subjected to the judgement of the experts, were made specific to the indicators that would be contained in the questionnaire. Table 2 displays these indicators.

Table 2.

Indicators selected by the panel of experts to assess the health of the adolescent lifestyle

Life style indicators
Physical activity
Diet
Relaxation/sleep
Substance abuse
Hygiene
Social networks/electronic games

The questionnaire, which was developed after analysing the types of habits that influenced each indicator in an obvious way, comprised 10 questions (some of the indicators included more than one question). The information collected to assess the different indicators is shown below:

• Diet: Questions were formulated referring to the frequency of ingestion of the different food groups, the amount of liquid, and the amount of soft drinks.

• Physical activity: Questions were asked regarding the hours of physical activity undertaken each day, in reference to recollections of the previous week. To discriminate the intensity, an indicative reference proposed in the Talk Test was used [69–72].

• Substance abuse: Questions focused on the consumption of alcohol, tobacco, cannabis, and other illegal drugs.

• Relaxation: A question was developed regarding the hours of sleep on school nights.

• Hygiene: The indicator included a question referring to the daily frequency of hand-washing and another referred to the daily frequency of tooth-brushing.

• Social networks and electronic games: The question asks to quantify numerically (in hours) the time spent each day communicating online with other people and playing electronic games, and school days and the weekend were differentiated.

Having assessed the comprehensibility of the questions in a sample of 67 adolescents, the final questionnaire was drafted, which consisted of 11 questions with 15 scoring items.

Reliability: internal consistency and temporal stability

To assess reliability (internal consistency and temporal stability), a total of 419 questionnaires were administered, of which 396 were valid. The values of α and stratified α were 0.66 and 0.77, respectively. The value of α did not vary significantly with the elimination of any item. A detailed analysis of each item is shown in Table 3.

Table 3.

Corrected item-total correlation (significance) and α if an item is eliminated

	Corrected item-total correlation (significance)	α if item deleted
P1	0.196 (p < 0.001)	0.65
P2	0.358 (p < 0.001)	0.63
P3a	0.304 (p < 0.001)	0.64
P3b	0.376 (p < 0.001)	0.63
P4	0.282 (p < 0.001)	0.64
P5a	0.361 (p < 0.001)	0.63
P5b	0.227 (p < 0.001)	0.65
P6	0.143 (p < 0.01)	0.66
P7	0.201 (p < 0.001)	0.65
P8	0.423 (p < 0.001)	0.62
P9a	0.399 (p < 0.001)	0.62
P9b	0.393 (p < 0.001)	0.64
P10	0.187 (p < 0.001)	0.65
P11a	0.138 (p < 0.01)	0.66
P11b	0.153 (p < 0.01)	0.66

The second round of questionnaires yielded 253 responses, of which 228 were valid and could be correlated with the first round. The ICC for the total scores was 0.860 (95 % confidence interval, 95 % CI; 0.82–0.89). Calculated values for each item are presented in Table 4.

Table 4.

Intraclass correlation coefficient (ICC) for each item

	ICC	95 % CI
P1	0.54	0.43	–	0.63
P2	0.85	0.80	–	0.88
P3a	0.77	0.71	–	0.82
P3b	0.86	0.81	–	0.89
P4	0.80	0.75	–	0.85
P5a	0.70	0.62	–	0.76
P5b	0.59	0.49	–	0.68
P6	0.68	0.59	–	0.75
P7	0.84	0.79	–	0.87
P8	0.95	0.93	–	0.96
P9a	0.88	0.85	–	0.91
P9b	0.66	0.58	–	0.74
P10	0.76	0.69	–	0.81
P11a	0.73	0.66	–	0.79
P11b	0.79	0.73	–	0.84

Content validity, criterion, and construct

The Kaiser-Meyer-Olkin (KMO) measure of the sampling adequacy was found to be 0.693, and Bartlett’s test concluded that the hypothesis of sphericity could be rejected (p < 0.001). These two values confirmed the appropriateness of performing an exploratory factor analysis (EFA). Five components with eigenvalues greater than 1, explaining 59.55 % of the variance, were extracted. The first component included items that questioned the amount of moderate-intensity physical activity performed every day of the week and was designated as physical activity. The second component included items that asked about the consumption of tobacco, alcohol, and other drugs and was designated as substance abuse. The third component, which included two elements related to the use of the Internet and electronic games and to hours of sleep, was designated as Rational Use of Technological Leisure (RUTL). The fourth component, containing questions related to the frequency of hand-washing and tooth-brushing, was designated as hygiene. The last component contained a question about the order of consumption of different types of nutrients and the frequency of intake of fluids and non-alcoholic beverages. This last component was designated diet. Table 5 summarises this analysis. The weight of each element in the corresponding factor is shown, as is the IFFS of each component.

Table 5.

Exploratory factor analysis. Extraction method: principal components with varimax rotation. Loadings of the rotation matrix and the IFFS are presented

Item	Component
	1	2	3	4	5
	Physical activity	Substance abuse	RUTL	Hygiene	Diet
P3b	0.93
P3a	0.89
P4	0.87
P8		0.84
P9a		0.83
P2		0.70
P9b		0.51
P5b			0.69
P5a			0.63
P10			0.58
P11b				0.74
P11a				0.63
P1					0.72
P6					0.60
P7					0.55
IFFSa	0.95	0.96	0.7	0.7	0.91

^aIFFS: Index of Fit for Factor Scales

The correlation between the total score on the questionnaire and that obtained in KIDSCREEN-10 was 0.21 (p < 0.001). With respect to the association of questionnaire scores with SRH, it was found that the total scores of the VISA-TEEN questionnaire were significantly different depending on the category manifested in the perceived health status. Figure 1 shows these results, and Tables 6 and 7 show the results of the ANOVA and post hoc tests, respectively.

Fig. 1.

Total VISA-TEEN based SRH

Table 6.

Descriptive and ANOVA test of the VISA-TEEN total scores in each category of the SRH

	n	(DE)	95 % CI			p
Perceived health (SRH)
Excellent	38	35.65 (5.48)	33.89	–	37.40	<0.001
Very Good	145	34.30 (5.26)	33.45	–	35.16
Good	139	32.18 (5.52)	31.24	–	33.11
Fair	28	26.96 (5.83)	24.78	–	29.14
Poor	3	18.33 (10.50)		n/a

n/a: not applicable; p: ANOVA test of significance among five categories

Table 7.

Contrasting post hoc VISA-TEEN total categories depending on their SRH

(I) SRH	(J) SRH	Mean difference VISA-TEEN total (I-J)	95 % CI			p	Cohen’s d
Excellent	Very Good	1.34	−1.44	–	4.12	ns	-
	Good	3.46	0.67	–	6.26	<0.01	0.63
	Fair	8.68	4.90	–	12.46	<0.01	1.53
	Poor	17.31	7.95	–	26.67	<0.01	2.06
Very Good	Good	2.12	0.29	–	3.95	<0.01	0.39
	Fair	7.34	4.21	–	10.47	<0.01	1.32
	Poor	15.97	6.85	–	25.09	<0.01	1.92
Good	Fair	5.21	2.07	–	8.35	<0.01	0.91
	Poor	13.84	4.72	–	22.97	<0.01	1.65
Fair	Poor	8.63	−0.87	–	18.10	ns	-

ns = no significance

After analysing the association between the scores of the different components of the VISA-TEEN and the variables sex, age, and FAS II, it was concluded that age was negatively associated with all components except hygiene, which is associated with sex. Physical activity is associated with both age and sex. None of the components showed significant association with purchasing power (FAS II). The provenance of the adolescent was determined to not be a confounder, and therefore, it was not necessary to adjust the results based on this variable. Table 8 summarises these results.

Table 8.

The association between different components and age, sex, and FAS II (quantitatively measured and categorised into three levels: low, medium, and high)

	Regression coefficient (p)
Component	Age	Sex	FAS	FAS categorised
Diet	−0.058 (0.002)	ns	ns	ns
Substance Abuse	−0.151 (<0.001)	ns	ns	ns
RUTL	−0.129 (<0.001)	ns	ns	ns
Hygiene	ns	0.261 (<0.001)	ns	ns
Physical Activity	−0.142 (<0.001)	−0.639 (<0.001)	ns	ns

Sex code: 0 = boy, 1 = girl

The significance of each coefficient is shown in parentheses; ns = no significance

The discriminatory power of the instrument, assessed from the modified Ferguson’s δ coefficient, was 0.972.

Discussion

The analysis of the different instruments and questionnaires used to assess the lifestyles of adolescents, both globally and nationally or locally, has been shown that there are few instruments that assess lifestyle healthiness using a summary score that takes into account all components of this study. Moreover, few are validated in Spanish, and there are none for the adolescent population. The VISA-TEEN rates adolescents’ lifestyle quantitatively and includes 5 components: Physical activity, Substance abuse, RUTL, Hygiene and Diet. These components can be assessed separately with other validated questionnaires [51, 73, 74] and are also mentioned in other questionnaires [2, 50, 75]. Unfortunately these last tests are not rated.

Questionnaire development phase

After the Focus Group transcription were analysed, the relevant variables to evaluate adolescents’ were: nutrition, physical activity, substance abuse, relaxation, hygiene, and the use of technology for communication or for leisure. No questions related to sexuality were considered because the data on the initiation of sexual intercourse in Catalonia indicate that adolescents initiate sexual activity at 15.8 years of age for girls and 15.7 years of age for boys [76]. In secondary school, 65.3 % of boys and 72 % of girls had not engaged in sexual intercourse [77], and thus, this component was not included because it would be irrelevant to a large group of adolescents.

In the past five years, there are not published validations of questionnaires that have scored using all of these variables. However, there are some who value certain variables separately. In 2011, Wright et al. [78] validated the HABITS questionnaire to assess lifestyle in children 7–16 years of age. It consists of 19 items with closed-response questions related to diet (frequency of consumption of different types of food and drinks) and the time spent watching television and playing electronic games. In 2012, Muchotrigo validated the Healthy Lifestyle Questionnaire (Cuestionario de Estilo de Vida Saludable, in Spanish) [79] for university students. In this case, 26 items with Likert-type responses are grouped into the following three components: sports activities, diet, and relaxation/sleep. Other studies have not validated instruments to assess lifestyle but may include this variable in some of analyses based on other questionnaires. In 2009, Vereecken et al. [80] studied the relationship between breakfast habits and lifestyle in adolescents 11 to 15 years of age from 45 countries. To assess lifestyles, questions were based on issues relating to substance abuse (alcohol and tobacco), physical activity, hours watching television, and eating habits (consumption of vegetables, fruit, and soft drinks) that appear in the HBSC.

Furthermore, other questionnaires that assess lifestyle include other variables. In 2012, Taymoori et al. published the validation of the Healthy Lifestyle Questionnaire (HLQ) for Iranian adolescents [49]. The final version consisted of 36 items drawn from different questionnaires on lifestyle of adolescents that are grouped into the following six factors: life appreciation, health responsibility, nutrition, social support, physical activity, and stress management. In 2013, Dinzeo et al. validated an abridged version of the Lifestyle and Habits Questionnaire (LHQ) for young (18–25 years of age) university students in the United States. The original version of LHQ consisted of 80 items, and its validation was published in 1998 [81]. The new version contains 42 items grouped into the following eight factors: physical health and fitness, psychological health, substance abuse, nutrition, environmental awareness, social awareness, accident prevention, and the meaning of life. Despite their multidimensional approach, both of the questionnaires were noted to be missing questions related to topics that are important for teens, such as the use of the Internet to interact and communicate. Both questionnaires (HLQ and LHQ) use Likert-type responses.

In the four questionnaires mentioned above, questions always have closed responses. The VISA-TEEN also contains closed-response questions but includes open-response quantitative questions and one ordered-choice question as well. Open-response questions of a quantitative type permit more accurate information to be obtained about variables for which it is more important to know a number (hours, cups) rather than an approximate interval or a qualitative assessment of frequency of performance or consumption. Other questionnaires that also use open-response questions are of a quantitative type. Some examples include the International Physical Activity Questionnaire (IPAQ), which poses questions concerning time in hours and minutes devoted to performing different types of physical activity during the past seven days [73], and Systematic Interrogation of Alcohol Consumption (Interrogatorio Sistematizado de Consumos Alcohólicos, in Spanish) (SALGA), Department of Health of the Government of Catalonia [82], which poses questions related to the number of standard drink units (SDUs) consumed in one week.

The analysis of the items in this study showed that all had good comprehensibility. The item that was rated as the most difficult was item number 4, concerning moderate and intense physical activity performed each day in reference to the previous week. The mean and median difficulty of this item were 3.11 and 3 points, respectively, based on a scale ranging from 0 to 10 (10 is the maximum difficulty). Qualitative inputs made by some of the subjects about this item suggested it should be divided into two parts. Additionally, examples of each type of activity were added to the final version of the questionnaire, thus facilitating the response process. All other items showed an average difficulty from 0.29 (question 9 concerning hand-washing and tooth-brushing) to 0.93 (question 1 related to food intake). In the latter case, it was considered especially important for this question to present a low difficulty, as ordered response questions tend to be difficult to answer [83]. In this item, this type of question was considered most suitable for assessing the adequacy of consumption of different types of food reported by adolescents with respect to the food pyramid. The pyramid, in its latest version (2012), does not specify the portions of different types of foods to consume but rather establishes an order of consumption (daily, weekly, and occasional) [84].

The average response time was 19.2 min. This duration is less than the maximum of 30 min recommended for studies where an interviewer applies the questionnaire [85]. It is similar to other questionnaires presented to school groups in European studies, such as KIDSCREEN-52, which requires 15 to 20 min to complete [58].

The diversity in the types of questions could complicate the comprehensibility and the response process by adolescents. To avoid these errors in the response process, and responding to suggestions made by some subjects who participated in the evaluation of comprehensibility, questions were ordered by type of response (ordered-choice followed by quantitative-response followed by closed-response) and instructions for answering each type were provided.

Validation phase of the questionnaire

The sex distribution (45.6 % girls/54.4 % boys) did not differ significantly from that of the general population (48.4 % girls/51.6 % boys) [86], and this age distribution was expected because after 16 years, the age of completion for compulsory education, the number of adolescent students decreases.

The distribution of the different spending levels according to the categorisation made using FAS II showed that more than three-quarters of the adolescents were classified in the "high" group. This apparent imbalance is because the proposed break points date from 2002, and there are indicators in FAS II that have changed substantially since then. The number of computers, for example, has increased in every household. This imbalance can be a source of bias that must be kept in mind when interpreting results where purchasing power is a factor. A study on the relationship between dietary habits of adolescents and purchasing power reaches the same conclusion and states the need to develop new, appropriate, and specific indicators to assess the socioeconomic status of adolescents [87].

Reliability: internal consistency

Regarding reliability, the value of both α and stratified α is above 0.65, thus demonstrating acceptable internal consistency that permits the use of the questionnaire in descriptive population studies, the objective for which the questionnaire was developed. The analysis of the characteristics of each item helped to confirm that there were none that needed to be removed to significantly increase the value of α. The corrected item-total correlations ranged between 0.138 and 0.423. Because the correlation was significant in all cases, and after verifying that the removal of any item did not improve the value of α, it was decided to retain all items for the final version. Even items that had less than a 0.20 (but significant) correlation remained in the questionnaire because it was decided that they provided interesting and necessary lifestyle information about adolescents.

The reliability (temporal stability) studied using the ICC obtained for the lifestyle total score demonstrated a very good agreement between the two occasions. In the individual analysis of each item, values ranged from 0.54 (good agreement) to 0.95 (very good agreement). Therefore, the items studied were accepted. Content validity was validated by the theoretical analysis, the involvement of stakeholders, and the classification conducted by the experts in the development phase of the questionnaire.

In terms of construct validity (EFA and IFFS), from the assessment of exploratory factor analysis, five components were extracted. Four were expected, and each contained items that were conceptually related (physical activity, substance abuse, hygiene, and diet). A fifth component, which was designated as RUTL, contained items related to entertainment technology and sleep. The relationship between these two variables and confirmation that they can be studied within a single component is supported by studies demonstrating the relationship between these variables. In 2013, Spies, Shapiro and Margolin analysed the existing evidence on the relationship between the use of social networking and psychosocial development of adolescents. Among other consequences, the authors found a relationship between intensive computer use, including online communications, and hours and quality of sleep [88]. The same year, Wolniczak et al. found a dependent relationship between Facebook and sleep quality [89]. Also in 2013, Don et al. concluded that excessive Internet use negatively influences health because of its relationship with few hours of sleep [90].

The IFFS values were acceptable for the RUTL and hygiene components and very good for physical activity, substance abuse, and diet. Therefore, the items assigned to each component are adequate, and the components are sufficiently independent from each other to allow for separate analysis.

For construct validity (hypothesis testing), first, the relationship between the total score for VISA-TEEN and that obtained for KIDSCREEN-10 was assessed. The correlation coefficient between the scores on the two questionnaires was r = 0.21 (p < 0.001). This correlation, though weak, is significantly different from 0 and is positive. Additionally, it is superior to the one that presents the same KIDSCREEN with physical health measurement offered by the Child Health Questionnaire (CHQ), which is r = 0.15 [58].

Second, the association between the total score on the VISA-TEEN and the assessment of perceived health was checked using the SRH. Scores were found to be significantly different when they were ordered by group, observing that the best scores for lifestyle corresponded to adolescents who reported excellent health, and the worst scores were for those who reported having poor health. In the intermediate group, scores for VISA-TEEN diminished when perceived health worsened. Post hoc tests showed that there were differences between all groups except between "Excellent" and "Very Good" and between "Fair" and "Poor". A relationship between some of the components of lifestyle and SRH was also found in the study conducted in Spain by Giron in 2012 [91], which concluded that diet and substance abuse influence the perception of health in young people, and in the study conducted in Greece in 2011 by Darviri et al. [92], which concluded that the factor most correlated with low SRH in adolescents is little physical activity.

Finally, the relationship of the various components of lifestyle with age, sex, and purchasing power was demonstrated. All components except hygiene were negatively associated with age (older, lower score). This tendency for declining health with age coincides with that found in other studies. Thus, in the case of physical activity, the HELENA study conducted throughout Europe shows how the average hours of daily physical activity diminishes with age (2 h at 13 years of age, 1.4 h at 17) [93]. With respect to substance abuse, increased risk behaviour with age can be found in both the HBSC-2010 worldwide [94] and in the local-level FRESC-2012 [75]. As for diet, Diaz and Trave found that the KidMed rate of adherence to the Mediterranean diet diminishes with age [95]. Additionally, both the HBSC-2010 and the FRESC-2012 show that consumption of fruit and the percentage of adolescents who eat lunch daily decreases with age. As for entertainment technology, results from FRESC-2012 show that the time spent chatting increases from 13 to 16 years of age, and then decreases slightly. The relationship with purchasing power was not significant for any component. This is inconsistent with other studies, such as HBSC-2010 [2], which did find an association between some of the components studying lifestyle and purchasing power. This fact could be due to the statement previously mentioned above, namely, an update of the criteria used to establish the socio-economic levels may be necessary because such an update has not been performed since 2002.

According to the way that EMPRO specifies assessing the quality of questionnaires results perceived by patients [57], criterion validity must be evaluated when there are shortened versions of existing validated questionnaires. This is not a prerequisite for new questionnaires, as it often happens that other validated measures assessing the same construct (gold standard) do not exist in order to make the comparison. In our case, we did not have a validated instrument that we could use as a criterion, and therefore, the criterion validity was assumed to be reaffirmed by the construct and content.

With respect to discriminatory power, the value of δ = 0.972 indicates that the questionnaire provides good discrimination. The KIDSCREEN questionnaire, which assesses HRQoL, provides discriminatory power between 0.94 and 0.98 in different versions of 52, 27, or 10-items [58].

Conclusions

The VISA-TEEN questionnaire developed in this study to assess the lifestyle of Catalan adolescents can be considered valid for its application in this population group (Additional files 1 and 2). It includes the following five components: diet, substance abuse, RUTL, hygiene, and physical activity. In addition, it has been shown to be quick and easy to answer, which bodes well for applicability.

Validity and reliability results show that this can be a good instrument to evaluate adolescents’ lifestyle, and can also be used to understand the role of how lifestyle influences adolescents. Moreover, it will also be useful to evaluate the efficacy of campaigns specifically designed to improve their lifestyle.

Abbreviations

WHO

The World Health Organization

HBSC

Health behaviour in school-aged children

HPLP II

Health promoting lifestyle profile II

PLQ

Personal lifestyle questionnaire

SAC

Scientific advisory committee

PRO

Patient-reported outcome measures

SRH

Self-rated health

FAS

Family affluence scale

RUTL

Rational use of leisure technology

HLQ

Healthy lifestyle questionnaire

LHQ

Lifestyle and habits questionnaire

SDUs

Standard drink units

CHQ

Child health questionnaire