Development, initial content validation and reliability of Nigerian Composite Lifestyle CVD risk factors questionnaire for adolescents

Abstract

Background

Cardiovascular disease risk (CVD) factors affect every age category including adolescents in developing nations. Prevention strategies are effective only when there are epidemiological data for the targeted populations. The collection of such data is only made easy with composite lifestyle CVD risk factors measures that are culturally sensitive and acceptable among the target populations.

Objective

The objective of the study was to develop a culturally sensitive and friendly composite lifestyle CVD risk factors questionnaire for adolescents in Nigeria

Methods

A systematic review was conducted to identify existing, published questionnaires from which items could be selected. Content and face validation were conducted using an expert panel and a sub-sample of the target population. Data was analyzed qualitatively and reliability was assessed using intra-class correlation and Kappa statistic.

Results

Based on the comments received from experts, the questions were restructured, simplified, clarified, formatted, some questions were added and expert reached a consensus. Kappa showed fair to moderate agreement in 65% of the questions and perfect agreement in one question.

Conclusion

The CVD risk factors questionnaire has acceptable content validity and reliability and should be used to assess CVD risk factors among adolescents in Nigeria

Introduction

Cardiovascular disease (CVD) is a global problem and is responsible for one tenth of disability-adjusted life years in low and middle-income countries and 18% in high-income countries. CVD is a notable cause of mortality in developing countries[1] and a marked increase is expected over the next few years[1,2] The burden of CVD in developing nations is attributed to increased longevity, urbanization and lifestyles changes[1]

Lifestyle related risk factors are associated with CVD mortality. Many of the lifestyle related risk factors for CVD are observed in adolescents[3,4]. Childhood and adolescent behaviors such as dietary habit, smoking and alcohol use typically extend into adulthood [1].

During adolescence, dietary habits and risky behaviors, such as smoking and drinking, are experimented and often life-long patterns are established [5–7]. The importance of adolescent health and its role in the burden of CVD is the focus of attention globally[8,9]. It is advocated that adolescents should be targeted for CVD prevention programs[10–12].

There is the need to establish the CVD risk profile of youths from developing nations particularly regions where little is known about CVD since most CVD preventative programs are confined to developing nations. Variations in cultural and society related risk factors and beliefs, particularly in rural areas and paucity of information among adolescents and a lack of studies which consider socio-cultural factors associated with CVD risk factors among this group[13] imply that existing programs cannot be easily transposed to developing countries . There is thus a lack understanding of the range of risk factors that expose adolescents to CVD.

Traditionally modifiable, lifestyle risk factors have been assessed individually[14–16]. According to Glassgow et al[17] each of the CVD risk factors is a risk factor for some other illnesses and chronic diseases and mortality such as diabetes, and cancer. This may explain why individual risk assessment is common. Planning a preventive or intervention program( primordial prevention, primary prevention and secondary prevention) for CVD needs consideration of the composite risk factor profile. Assessment of each lifestyle CVD risk factor with an instrument will make population based studies and intervention among this sub-population a herculean task. The authors conducted an extensive literature search using seven databases from inception of each data base to 2012 and found no published CVD risk factors questionnaire developed in Nigeria either for adult or adolescents population Therefore, the development of composite lifestyle CVD risk factors measure for adolescent population is advisable . Careful consideration is however required to ensure that the composite risk factors assessment tool is culturally sensitive and acceptable to adolescents in their respective locations and culture. To our knowledge and to date there is no composite validated measure that assesses lifestyle CVD risk factors for adolescents in Nigeria. The aim of this study was to develop an appropriate and culturally acceptable CVD composite risk factor measure to investigate the prevalence of lifestyle CVD risk factors among adolescents in rural Nigeria.

Methodology

Ethical approval was obtained from the Stellenbosch University Health Research Ethics Committee, South Africa. Permission to conduct the study was obtained from local educational authority and principals of the selected secondary school for the reliability study. Adolescents and parents of adolescents were informed of the research and written/verbal consent was obtained from the parents and assent from the students. Purposive sampling was used to select 21 adolescents who were attending private schools and whose parents were graduates in order to have adolescents who were proficient in English language. They were between 15 and 18 years old. The study was conducted at the Department of Physiotherapy College of Medicine and places where those adolescents could be located

Item generation and design of draft questionnaire

A systematic review was conducted between August and September 2008 and updated in August 2012 using electronic databases accessible via the Stellenbosch University library: Pubmed, CINAHL, Psych Info, Proquest, Sport discuss and Cochrane as these are appropriate databases with studies related to the topic and the key words. The following key words were used; CVD, CVD risk factors, questionnaires, smoking, alcohol, drinking, physical activity and diet.

Thirty eight questionnaires were identified from the systematic review but eight questionnaires were selected based on four criteria which were adapted from recommendations in choosing measuring instruments from Glassgow et al¹⁷ Based on the outcomes of the systematic review, a draft composite lifestyle CVD risk factors questionnaire was designed using eight questionnaires namely ; 1) The Youth Health Survey by Gilmer et al[18] which was developed as part of school-based longitudinal study to assess and track behavior and attitudes related to CVD risk factors in children and adolescents. Psychometric properties of the subscales were as follows: Physical activity: internal consistency 0.74, test retest reliability 0.70; Diet: internal consistency: 0.84, test retest 0.79; Smoking : Spearman rho correlation : 0.89[18] 2). A New Measure of Smoking Initiation and Progression among Adolescents by Sun et al[19]. Content validity was done using adolescents. Test-retest reliability for overall sample was: 0.85, male 0.83, female 0.87[19]. 3) Alcohol Use Disorder Identification test (AUDIT) [20]. Studies show sensitivity of the AUDIT ranged between 0.6– 0.9, and high values of reliability. It has been validated in Nigeria among youths in the university[21]. 4) Instrument for Monitoring Adolescent Health Issues[22]. Questions show moderate to high test retest reliability in subscales. 5) Adolescent Smoking Uptake Continuum[23] is used for assessing smoking likely hood and established smoking. It has been found to have acceptable validity, reliability and sensitivity in various populations[23]. 6) Youth Risk Behavior Surveillance System Questionnaire(YRBSS) [24] for monitoring adolescent health. Studies have reported moderate to high validity and reliability[25,26]. 7) Physical Activity Questionnaire for Adolescents[27]. It was developed to measure the level of physical activity among high school adolescents. It has shown consistent high validity and moderate reliability. 8) Food Intake Questionnaire by Johnson et al, 2001[28]. Pearson correlation ranged from 0.42–0.76 for various food groups.

Design of the draft Nigeria Composite Lifestyle CVD

Risk Factors Questionnairefor adolescents

Demographic Items were taken from the Youth Risk Behavior Surveillance System (YRBSS). Items for the smoking subscale were taken from Smoking Uptake Continuum, Youth Health Survey and YRBSS . Itemsfor the alcohol subscale were taken from Alcohol Disorder Use Identification Test(AUDIT- 3 ) and another Item was added by authors. Items for the Physical activity subscale were taken from YRBSS, Physical Activity Questionnaire for Adolescents( PAC-A). Items for the Nutrition subscale were taken from Food Intake Questionnaire. Other items were added by authors. These Items were taken from these specific questionnaires because they were in line with the content coverage of the questionnaire being developed for example, smoking intention and likelihood were seen in smoking uptake continuum. Changes were made in different subscales by the expert panel and the questionnaire was formatted to improve its appearance and face validity.

Validation by expert panel

The draft questionnaire was given to a panel of experts to ascertain its face and content validity. The validation process in this study was conducted in line with the validation processes of questionnaires by Kingston et al[29] and John et al[30]. Particularly, validation of the questionnaire was carried out using guidelines stipulated by Beaton et al and Boyton et al[31,32]. Purposive sampling was used to recruit experts from Cardiology, Physiotherapy, Exercise Physiology, Nutrition and Psychiatry. Specialists from these departments at the University of Ibadan were asked for the names of academics who have worked extensively and published in the subject area of this research. These experts were professionals who have published at least one international paper in the area of selected CVD risk factors assessment, questionnaire development, cross-cultural validation or validation studies in the last five years. Other experts who had not published at the time of the study were specialists and professionals involved in lecturing/clinical consultancy and research supervision at postgraduate level. The purpose of the expert panel was to ascertain the utility and applicability of the CVD risk factor questionnaire among Nigerian adolescents in terms of language appropriateness, knowledge of activity and experiences. Nigeria is a multiligual developing country in Africa with over 150 million people of which 22% are adolescents. Nigerians are not native speakers of English language though English language is the country's official language. Therefore questionnaires developed in English for Nigerians need to be culturally adapted more so items were selected from previous questionnaires that were not developed by Nigerians or for Nigerians. The experts carried out cultural adaptation of the English version of the questionnaire using the guidelines by Beaton et al[31] and checklist for questionnaire development by Boyton et al[32] to ascertain that there was no ambiguity or double-barrel question and that questions in the questionnaire covered all the content areas. The Beaton et al[31] consider the following issues; semantic equivalence, idiomatic equivalence, experiential equivalence and conceptual equivalence.

At the panel meeting, the first author, who was the convener of the meeting explained the purpose of the meeting, the purpose of the questionnaire- for monitoring, what dimension each subscale of the questionnaire addressed as there are various dimensions in all the subscales because they were complex health phenomena. The process of validation was done according to subscale subscale, each subscale being handled by the expert in that field and contributions by other experts were encouraged.

Following the consensus ( based on relevance of the questions to objective of the questionnaire and language appropriateness) of the experts after the meeting, a penultimate version was developed with all the amendments, clarifications, simplications, additions and formatting effected. This version was then used for further content validation (pre testing) among the adolescent group.

Validation by adolescents (pre-testing)

The pre-testing was conducted in the target population in Ibadan (Ibadan North Local Government Area of Ibadan City in south-western Nigeria). Twenty one adolescents were selected using sample of convenience ( adolescents in teenagers's class of a church) participated in the pre-testing procedure of the questionnaire. They were male and female Nigerian adolescents of 15–18 years who were proficient in English language. The adolescents were informed of the purpose of the pretest and validation process. All volunteer adolescents were given the penultimate version and a feedback questionnaire. It was decided that the feedback questionnaire would be used and not a focus group in order to allow the adolescents to express themselves freely. The semi-structured questionnaire was used to note any difficulties adolescents experienced in interpreting and completing the penultimate version of the CVD questionnaire. Adolescents were asked to note specific questions/sections that they had difficulty with, questions they thought were not necessary and questions they thought should be added. Adolescents were requested to write their suggestions in the space provided on the feedback questionnaire. The semi-structured feedback questionnaire and the penultimate version of the CVD risk factors questionnaire were collected for analysis of content.

Reliability

The reliability of the questionnaire was conducted in a secondary school at Egbeda Local Lovernment Area Oyo state, Nigeria. One hundred and fifteen adolescents from a public secondary school participated in this study. All adolescents between 15 and 18 years in the school were invited to the study. The 115 adolescents were all consenting participants of about 150 adolescents in the age group studied. The participants were given the questionnaire to complete and 48 hrs afterwards another copy of the questionnaire was given to each participant. ‘Seven days’ interval or more as usual in other studies[33] was not appropriate in this study because the questionnaire had a seven- day recall i.e the participants were asked what they did or ate in the last seven days.

Data analysis

The data for the content validation was analysed qualitatively using content analysis. This method is very useful in analysis of semi-structured questionnaires like the one used for feedback from the adolescents. Test retest reliability using intraclass correlation (ICC) for the five subscales was carried out. The interpretation of the ICC”s were as follows: 0–0.2 is poor, 0.3–0.4 is fair, 0.5–0.6 is moderate, 0.7–0.8 is strong or very good correlation and >0.8 is almost perfect correlation[34] In order to use ICC, some items of the questionnaire were scored to determine specific CVD risk factors. Scoring was done for every subscale but there was no overall score because the questionnaire is a profile scale. Kappa statistics was also used to assess percentage agreement between the first and second administration of questionnaire

Results

A total of 38 questionnaires were identified out of which eight were selected for aggregation of questions because they met the criteria the authors had set. Nigeria Composite Lifestyle CVD risk factors Questionnaire for Adolescents was developed (see appendix). The questionnaire is a 33 item profile scale with six subscales which are; demographic subscale, CVD indicators subscale, smoking subscale, alcohol subscale, physical activity subscale and nutrition subscale. The scales of measurement are nominal and ordinal scales. Scores for various subscales were determined to assess the prevalence of each CVD risk factor. ICC ranged between 0.3 – 0.7 in various subscales. Kappa shows moderate agreement in more than 65% of the questions Kappa statistics showed fair tomoderate agreement in 65% of the questions in subscale2–6, perfect agreement in one question, slight agreement in 13% of the questions and slight disagreement in13% of the questions., this is shown in table 3. Nutrition subscale has 5 sections indicated as 30a, 30b, 30c, 30d,30e

Table 3.

Kappa statistics for all items in CVD indictors, smoking, Alcohol, physical activity and nutrition

QUESTIONS	%AGREEMENT	KAPPA VALUE
5	67.8	0.2
6	77.9	0.5
7	79.2	0.3
8	94.7	0.5
9	95.5	-ve
10	63.2	0.3
11	93	0.2
12	94.7	0.12
13	74.3	0.3
14	99.3	-ve
15	98.3	-ve
16	95.7	-ve
17	95.7	0.3
18	98.3	0.5
19	70	-ve
20	98.3	0.3
21	88	0.1
22	99	0.9
23	49.6	0.1
24	55	-ve
25	51	0.2
26	51.3	0.2
27	64.8	0.5
28	50.5	0.2
29	51.3	0.3
30a	35.8	0.1
30b	31.8	0.1
30c	37.6	-ve
30d	39	0.2
30e	44	0.2
31	75.4	0.5
32	67.2	0.5
33	72.2	0.4

Discussion

This study was carried out to develop an instrument called “Nigeria Composite Lifestyle CVD Risk Factors Questionnaire For Adolescents” for monitoring adolescents' Cardiovascular health. Nigeria like most developing countries suffers the double burden of infectious and non communicable disease. However due to lack of resources, the country is not fully awake to solving the problem of CVD which is highly prevalent. Adolescents' present lifestyle may result in CVD epidemic in future as such there is the need to monitor adolescents heart health.The development of the instrument went through various rigorous stages.The questionnaire addresses basic behavioural issues in adolescents lifestyle which constitute CVD risk factors . The dimensions assessed in this questionnaire in relation to CVD and monitoring of adolescents heart health were smoking likelihood, established smoking, harzardous drinking, dietary pattern; fruit and vegetable intake, salt intake and food preference, physical activity level and sedenary living, physical education in school curriculum, practical physical education and CVD indicators in lifestyle of adolescents which will help in monitoring adolescents heart health and evaluate intervention programs.This is the first questionnaire developed to assess and monitor adolescents heart health in Nigeria. The Nigeria CVD risk factors questionnaire is meant to assess CVD risk factors profile of adolescents and not CVD risk. CVD risk is done by computation of certain risk factors including serum lipids which is not considered in this questionnaire. This questionnaire will enable clinicians assess CVD risk factors profile of adolescents and clustering of such CVD risk factors among adolescents

Validation

Face and content validation of the questionnaire are acceptable having gone through expert panel and target population i.e the adolescents . This implies that the relevant content areas were covered and there was no ambiguity in questions. This is in agreement with Terwee[38].Internal consistency of this questionnaire could not be assessed because each subscale looks at many dimensions [38] for example, smoking subjectscale assesses smoking likelihood/ intention, established smoking and smoking volume. Each of the dimension is important especially in development of CVD prevention program for adolescents to address specific issues.

Adolescent comment on adding questions on sex shows that adolescents' sexual health in Nigeria needs to be given urgent attention.The comment also shows adolescents lack adequate knowledge on reproductive health however this is beyond the scope and objective of this questionnaire. It could also be that adolescents do not have adequate knowledge of CVD risk factors. Knowledge of a disease condition and experiences by expert/target population are relevant issues and are very necessary for validation of an instrument[38]. Feedback of adolescents during pre-test could inform the researcher about the knowledge of adolescents regarding the condition. Adolescent's feedback of inclusion of questions on sex may indicate that the adolescent does not have adequate knowledge of CVD risk factors or that the adolescent could just be curious. However, since adolescents were given feedback questionnaires and Nigeria composite lifestyle CVD risk factors questionnaire, they were expected to make their suggestions in relation to content coverage which they had been informed. Failure to do this adequately could be attributed to poor knowledge of condition in question, religious bygotry as in the case of alcohol or curiosity to know other things important to adolescents like sexual and reproductive health. An adolescent's suggestion to remove all questions In alcohol subscale suggests that some Nigerian adolescents may not be knowledgeable about CVD risk factors. We also think this adolescent may be very religious as this response could be a product of his/her religious inclination which is a common feature among Nigerians generally. However, according to Beaton et al, authors can only revise a question when 2 or more adolescents have problem with a question therefore we did not revise the questions based on one adolescent's comment.

Poor understanding of adolescents about CVD, could have negative influence on content validation if only the target population were involved in the validation process. However the process of development involved literature review , expert panel and adolescents as such validation process was rigorous enough to produce a valid questionnaire, this is supported by Terwee[38]. Another important issue raised by an adolescent is temptation to smoke. This we interprete to mean in our cultural context peer pressure. We opine that there is a need to assess influence of peer pressure in adolescents' risky behaviour. This has been supported by various studies on why adolescents smoke[35–37]. However the present questionnaire was not developed to capture this important problem but we recommend this for subsequent questionnaire.

Reliability

The reliability ranged from slight to strong agreemen. Kappa showed very good percentage agrreement in majority of the questions, however kappa was relatively low and moderate for many questions. This is in agreement with Korean YRBSS and CDC reliability studies of YRBSS[24]. Some questions had negative kappa values which could be genuine. Many items in nutrition and physical activity showed low kappa values. This we think may be due to rural adolescents feeding habits and physical activity pattern not beeing stable because of poverty level. In rural communities daily feeding depends on how much a person has as such feeding pattern may not be established. We opine that food diary may be a more reliable method of assessing nutrion and physical activity log may be very useful in assessing physical activity. Generally reliability studies in adolescents shows that to have high reliability value for questionnaires, adolescents must perceive what is being done as important to ensure that they do not complete the questionnaire in a hurry, thereby having many errors or deliberately giving false information. There may be need to conduct other reliability studies of this questionnaire in urban and other rural areas of Nigeria. Intraclass correlation ranged from fair to very good reliability in various subjectscales. Most stable response was in alcohol subscale. This could be because, most of the adolescents had never taken alcohol. Physical activity and Nutrition subcales were the least stable. This could be because their feeding pattern and physical activity pattern are determined by economic power and available facilities. Both Kappa and ICC show low values in these subscales. According to Terwee[38] ICC is considered important clinically if it is 0.7 . Reliability values using prevalance rates at 2 different times will be useful to further determine the reliability of the instrument since it is for monitoring heart health.

Availability of valid and reliable Nigeria composite lifestyle CVD risk factors questionnaire is very pertinent clinically and in epidemiology. With increasing prevalence of CVD in Nigeria, assessment of CVD risk factors among adolescents who come to clinic for other medical conditions will be enhanced. Population based study of adolescents cardiovascular health will also be enhanced. Findings from such hospital based studies and population based studies will lead to development of CVD prevention program thereby ensuring future heart health of Nigerians. This questionnaire can also be adapted in other African countries, particularly west Africa for monitoring adolescents cardiovascular health.

Conclusion

Nigeria Composite Lifestyle CVD Risk Factors Questionnaire for Adolescents is a valid and fairly reliable questionnaire to assess lifestyle CVD risk factors among adolescents in Nigeria. Further validation may be necessary to establish and improve psychometric properties of the questionnaire.

Recommendation

Further validation and reliability studies of this questionnaire is necessary

Translation of the questionnaire into local languages in Nigeria is important and will enhance its utility among rural adolescents in various parts of Nigeria

Table 1.

shows the Experts' in put in various subscales of the questionnaire

	Clarification	Amendment	Addition	Simplification	Formatting
Demographics		Changed to ‘yy’ for year, ‘mm’ for month, ‘dd’ for day. Q3 option ungraded changed to ‘other grades’			Formatting was done to improve the appearance by using Times New Roman, creating tables and boxes where necessary
Subscale 2	Q6 examples were given as: running, playing football and pounding to clarify Q7 Little exercise was clarified as: climbing the stairs and walking for ten minutes		Q9 a new question was added by the expert panel	Q5 exerted was replaced with ‘when you exercise”
Subscale 3		Q14 Condition for smoking in future such as; when you are grown up and or have money was added
Subscale 4	Introductory statement was changed from “liquor” to hot drinks with examples as ogogoro Q21 - standard drink was clarified in volume as: a bottle of coke(35cl) used in Nigeria.	Nos of the questions changed as there was an additional question in CVD indicators subscale.
Subscale 5		Q27 Do you have PE on your time table was restructured to read: practical PE in the time table (in the school curriculum) Q28 Do you have practical PE was restructed to read: practical PE on the field ( experience) not just in the curriculum Q29; How many sporting competitions do you take part in was restructured to read: how many sporting competition/inter house sport did you take part? with usual examples such as sprint, relay, football	Boxes were provided for all the various frequency of questions in the subject scale
Subscale 6		Experts requested that: a table be created for food frequency to make it easier for adolescents to complete that section Q30;Common breakfast cereal in Rural Nigeria was made the first on the list, ‘rice crispice’ was excluded because it is not a usual breakfast cereal in Nigeria, other food types were written with their product; yam and yam products instead ‘of listing alll the product of yam such as elubo, pounded yam, asaro. Cassava and cassava products’.

Table 2.

ICC for various subscales of the questionnaire

Subscale	ICC (95% CI)
CVD Indicators	0.65 (0.47– 0.75)
Smoking	0.5 (0.34–0.68)
Alcohol	0.7 (0.62–0.82)
Physical activity	0.46 (0.22–0.63)
Nutrition	0.3 (0.01–0 .50)