Abstract
Introduction:
The aim of this study was to validate the AIDA (Attention, Interest, Desire, and Action) Model-Based Antismoking Campaign Questionnaire to be used in Malaysian population.
Method:
This study consists of mainly translation, validation, and pilot testing. The translation phase using forward and backward translation, involved three panels and three translators. The validation was a cross-sectional study conducted from May to July 2023 with a purposive sampling technique. The data was collected through e-mails among eight experts. These experts answered an online questionnaire on a four-option Likert scale, based on the four concepts of relevancy, clarity, comprehensiveness, and representativeness. The content validity index (CVI) was measured on the scale of the content validity index (S-CVI/Ave) and Universal Agreement (UA). For pilot testing, the final version was tested among 25 non-smokers and six smokers to determine its reliability using the Cronbach’s alpha.
Results:
The content validity study for relevancy, clarity, comprehension, and representative S-CVI/Ave is 0.85, 0.79, 0.79, and 0.84, respectively. The CVI score of above 0.83 indicates all items are relevant and representable. The pilot testing shows high internal consistency for both samples, more than 0.85.
Conclusion:
In summary, the adapted translated version’s content validity index was satisfactory, and it can be further pilot tested among the other target population.
Key Words: Content validity, reliability, translation, AIDA model- anti-smoking campaign
Introduction
To engage, empower, and influence communities, health communication can be done through advertising, education, and health-related materials [1]. Health communication can also be defined as the study and implementation of communication strategies to inform and influence individuals. Advertising may take the form of native print advertisements or online advertisements [2]. As it can help raise knowledge about health issues and encourage healthy habits, advertising may help enhance health literacy and help people make the right health decisions [3, 4].
The development of activities and interventions aimed at positively altering behaviours is related to social marketing, which entails developing health communication campaigns that use advertising to help target audiences become more aware of health issues and motivate them to act [5]. Campaigns involving health communication can be used to encourage healthy habits, inform people about health concerns, and motivate them to seek medical assistance when necessary [6]. In recent years, advertising has been shown to be a potent instrument for enhancing public health outcomes and boosting health communication[5]. To be successful, advertising-based health communication strategies should be tailored to their target demographic and the health issue they are addressing.To create messages that resonate with the intended audience, theories and models might be applied [7].
By using effective approaches, health communication campaigns can effectively promote healthy habits and enhance public health outcomes using any relevant type of advertising. Through the use of advertising, numerous effective health campaigns have been carried out [8]. Mass media, social media, or physical materials are frequently used in health communication initiatives to provide messages that are intended to influence people’s health behaviours. Additionally, anti-smoking initiatives with persuasive, vibrant language, and advertisements may be beneficial in lowering smoking rates. Health communication initiatives through advertising can successfully encourage healthy habits and enhance public health outcomes by using effective techniques and resources.
The AIDA model, a well-known advertising framework that stands for Attention, Interest, Desire, and Action [2], describes the four stages of people’s decision-making process. In market research, the AIDA approach is commonly used to evaluate campaign success and identify the areas for improvement. By posing intriguing questions, this model effectively grabs the audience’ attention. A study involving eye-tracking techniques found that anti-smoking messages with graphic images and text were more successful at capturing smokers’ visual attention compared to those without [9]. This insight can be valuable for future efforts. This model can be used to assess the efficacy of the content used in anti-smoking programmes to encourage quitting or lowering the prevalence of smoking. In order to enhance engagement and participation, anti-smoking campaign questionnaires should include captivating questions and images. Anti-smoking initiatives are a topic on which the AIDA model is applicable. Anti-smoking campaign questionnaires should include questions that elicit emotional reactions because it has been demonstrated that emotional appeals are successful in driving behaviour change [10].
According to Saiful and Yusoff, [11] ,content validity is the degree to which the survey items accurately reflect the construct of interest and it is the initial stage of validating questionnaires. The questions must precisely assess the intended constructs and be pertinent to the campaign in order to be considered relevant. Examining the content of the questions and evaluating the response rate are two techniques to gauge a questionnaire’s validity. To be successful, advertising-based health communication strategies should be tailored to their target demographic and the health issue they are addressing.
Researchers must carefully review the questions to make sure they accurately measure the desired construct and that they are pertinent to the campaign if they are to measure the questions’ substance. Furthermore, there must be no bias in the questions, and they must be appropriate for the targeted population.
The purpose of this study was to assess the newly adapted questionnaire in terms of content validity and reliability. This questionnaire was intended to measure the efficiency of selected anti-smoking campaigns. Therefore, before using the questionnaire as a study tool, researchers should evaluate its validity and reliability. By doing this, researchers can ensure the questionnaire is reliable and valid to produce an accurate findings.
Materials and Methods
This study was guided by the translation validation guidelines from Wild et al. [12]. There were four phases to this research. This study started with the preparation phase and translation process, followed by the validation procedure and pilot testing as shown in Figure 1.
Figure 1.
Flowchart of the Study Process
Preparatory Phase
In translation phase, the initial process is to find the applicable questionnaires concerning the AIDA model. During the search, it was discovered that the questions from Kulkani et al. [13] research were appropriate [13]. Panel 1 of the validation translation group performs the adaptation procedure. The adaptation process is to comply with the anti-smoking advertising effort. The questions were originally in English but later translated into the Malay language. Then, the harmonisation and revision were done by two panels (Panel 1 and Panel 2).
Translation Phase
The amended version was given to Translator 1 and Translator 2, and they can proceed with the forward translation. The Translator 1 is a certified professional translator, and the Translator 2 has experience working as a translator with Malaysian Institute of Translation & Books (ITBM - Institut Terjemahan & Buku Malaysia) which is the National Translation Institute of Malaysia. After the two forward translations were completed, they were both harmonised. Panels 1 and 2 decided which translations were the best and most acceptable via Google Meet. After that, Translator 3, who had a first-degree in Teaching of English as a Second Language (TESL) accreditation from the University of Malaya (UM), completed the backward translation. In order to determine whether the translation is accurate and conveys the same content, Panels 1 and 3 then performed a contrast between the original and backward translation versions via an online meeting platform (for example, Google Meet).
Content Validation Phase
This is a cross-sectional study conducted from May to June 2023 applying a purposive sampling technique. The data was collected through e-mails from experts who were willing to participate. These experts responded to an online questionnaire on a four-option Likert scale, based on the four criteria included: relevancy, clarity, comprehensive, and representativeness. Eight experts from several related fields of exercise, psychology, public health, cognitive motor, clinical nursing, sport psychology, sport science, public health, and nursing. The eight numbers of experts participated in this study as suggested by Polit et al. [14]. All the indices as defined in Table 1.
Table 1.
Definition and Formula for CVI Indices
| Definition | Formula | |
| I-CVI (item level content) | Proportion given by expert for score 3 or 4 | I-CVI= (agreed item/ No of expert) |
| S-CVI | Average of I-CVI for all items divided by number of items or experts | S-CVI/ Ave= (sum of I-CVI/ sum No. of items S-CVI/ Ave= sum of proportion relevance rating / No. of Experts |
| S-CVI/UA | Proportion of items on scale with rating 3 or 4 by all experts. Universal agreement (UA) score is given as 1 when the item achieved 100% experts in agreement. | S-CVI/UA = sum of UA scores/ No. of items |
Note: I-CVI, item content validity index; S,scale content validity index; Ave, average; ( /), divided; UA, Universal Agreement. The Formula based on article [11, 15].
Prior to the calculation of CVI, the experts ratings of 1 and 2 were coded with 0, while the ratings of 3 and 4 were coded with 1.The item-level content validity index (I-CVI), scale-level content validity index based on the average method (S-CVI/Ave), and scale-level content validity index based on the universal agreement method (S-CVI/UA) CVI indices were used to assess content validity. Calculating an I-CVI score, requires all ratings agreeing to a rate of ‘1’ were averaged with the total number of experts. Thus, each item received an I-CVI score. In order to obtain S-CVI, all the total I-CVI of each item were averaged with the number of items which is 28. For the calculation of UA, all experts must obtain a score of 1 for the item, then the UA score is 1; meanwhile, if one of the experts scores 0 for the particular item, the UA is 0. To get the S-Ave/UA score, the total UA must be averaged and divided by the number of items.
Pilot Testing Phase
The outcome from content validation phase was later analysed and discussed. The comment from the expert panels were used to revise the final version before the pilot testing. For the final version, panels have agreed to separate the questionnaires based on smoking and non-smoking status. Then, the pilot testing was conducted among 25 non-smokers and six smokers, via Google Forms. Through the use of Cronbach’s alpha, the internal consistency and reliability of the questionnaires are to be evaluated throughout the pilot testing. This gauges how well the questionnaire’s items are linked to one another and yield consistent findings.
Results
Content validity
For content validity study, the determination of content validity involved eight experts consisting of six females (75%) and two males (25%). They are between 31 to 55 years old and the majority hold a Doctor of Philosophy (PhD – philosophiae doctor) (62.5%) and master’s degree (37.5%). All of them are university lecturers from Universiti Teknologi MARA (UiTM) (62.5%), International Islamic University Malaysia (UiAM – Universiti Islam Antarabangsa Malaysia) (25%), and Universiti Pertahanan Malaysia (UPNM) (12.5%).
Tables 2, 3, 4, and 5 show relevancy, clarity, comprehension and representative S-CVI/Ave which are 0.87, 0.83, 0.83, and 0.87 respectively. The CVI score above 0.83 indicates all items is relevant, clear, comprehensible, and representable (16). This result shows acceptable SVI/Ave. However, the average UA shows 0.21, 0.14, 0.17, and 0.24 respectively.
Table 2.
The Rating of “Relevancy” for each Item by the Experts
| Item label | Ex1 | Ex 2 | Ex3 | Ex4 | Ex 5 | Ext 6 | Ex7 | Ex8 | Expert Agreement | I-CVI | UA |
|---|---|---|---|---|---|---|---|---|---|---|---|
| A1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 5 | 0.67 | 0 |
| A2 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A3 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A4 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A5 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| A6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| A7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I3 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 7 | 1 | 1 |
| I7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 7 | 1 | 1 |
| D1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D3 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D6 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D7 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW3 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| AW4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 7 | 1 | 1 |
| AW7 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
Notes : A,action; I, Interest; D, desire; AW, awareness; Ex, Expert; S-CVI, 0.87; S-CVI/UA, 0.10
Table 3.
The Rating of “Clarity” for each Item by the Experts
| Item label | Ex1 | Ex2 | Ex 3 | Ex 4 | Ex5 | Ex6 | Ex7 | Ex8 | Expert Agreement | I-CVI | UA |
|---|---|---|---|---|---|---|---|---|---|---|---|
| A1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| A2 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 6 | 0.83 | 0 |
| A3 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 6 | 0.83 | 0 |
| A4 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A5 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| A6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| A7 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| I1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I3 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 1 |
| I4 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| I5 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| I6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I7 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| D1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D2 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D3 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 5 | 0.67 | 0 |
| D4 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D6 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D7 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| AW1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| AW2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW3 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| AW7 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
Notes : A,action; I, Interest; D,desire; AW, awareness; Ex, Expert; S-CVI, 0.83; S-CVI/UA, 0.11
Table 4.
The Rating of “Comprehensiveness” for each Item by the Experts
| Item label | Ex 1 | Ex2 | Ex3 | Ex4 | Ex5 | Ex6 | Ex7 | Ex8 | Expert Agreement | I-CVI | UA |
|---|---|---|---|---|---|---|---|---|---|---|---|
| A1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| A2 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A3 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A4 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| A5 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| A6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| A7 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 0 |
| I1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| I3 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.83 | 1 |
| I4 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| I5 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| I6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| D1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D2 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D3 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D4 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| D6 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| D7 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| AW1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.67 | 0 |
| AW2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW3 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.83 | 0 |
| AW6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 7 | 1 | 1 |
| AW7 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 6 | 0.67 | 0 |
Notes : A, action; I, Interest; D, desire; AW, awareness; Ex, Expert; S-CVI, 0.83; S-CVI/UA, 0.11
Table 5.
The Rating of “Representativeness” for each Item by the Experts
| Item label | Ex 1 | Ex2 | Ex 3 | Ex4 | Ex5 | Ex6 | Ex7 | Ex8 | Expert Agreement | I-CVI | UA |
|---|---|---|---|---|---|---|---|---|---|---|---|
| A1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.75 | 0 |
| A2 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 6 | 0.75 | 0 |
| A3 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.87 | 0 |
| A4 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 0.87 | 0 |
| A5 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.75 | 0 |
| A6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| A7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| I2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.85 | 0 |
| I3 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| I5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| I6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| I7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| D1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.75 | 0 |
| D2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| D3 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| D4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 6 | 0.75 | 0 |
| D5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| D6 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| D7 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| AW1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| AW2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| AW3 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 6 | 0.75 | 0 |
| AW4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| AW5 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 7 | 0.87 | 0 |
| AW6 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 |
| AW7 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1[i] | 6 | 0.75 | 0 |
Notes : A, action; I, Interest; D, desire; AW, awareness; Ex, Expert; S-CVI, 0.87; S-CVI/UA, 0.21
Pilot testing
The participants involved in pilot testing were around 18 to 28 years old. In total, 12 males and 19 females were involved in this study covering both smokers and non-smokers. Most of them are university students; the majority of them are from non-sports science faculty. For the smoking participants, they are light smokers with an average of four cigarettes a day (refer to Table 6). Table 7 shows the Cronbach alpha results for items in the four domains in the AIDA model which were action, interest, desire, and awareness. The internal consistency results among smokers show a score range of 0.85 to 0.93 for the four domains. For non-smoking participants, it showed a range of 0.88 to 0.97 internal consistency.
Table 6.
The Sociodemographic Characteristics of Subjects Participant in the Pilot Study
| Non-smoker (n=25) n (%) |
Smokers (n=6) n (%) |
|
|---|---|---|
| Age | ||
| 18-23 | 24 (96) | 5 (83.3) |
| 24-28 | 1 (4) | 1 (16.7) |
| Gender | ||
| Male | 6 (24) | 6 (100) |
| Female | 19 (76) | 0 (0) |
| Semester | ||
| 1-3 | 15 (68) | 3 (50.1) |
| 4-7 | 10 (32) | 2 (3.4) |
| Not related | 1 (16.7) | |
| Faculty | ||
| FSR | 8 (32) | 5 (83.3) |
| Non-FSR | 17 (68) | 1 (16.7) |
| Smoking duration | NR | 4.00 (1.8) |
| Puff/day | NR | 3.5 (1.6) |
Notes:NR, not related
Table 7.
The Cronbach Alpha for Each Domain in Different Samples.
| Domain | Non-smoker | Smokers |
|---|---|---|
| Action | 0.85 | 0.88 |
| Interest | 0.85 | 0.95 |
| Desire | 0.89 | 0.97 |
| Awareness | 0.93 | 0.96 |
Discussion
The CVI for this study was 0.87 for relevancy, 0.83 for clarity, 0.83 for comprehensiveness, and 0.87 for representativeness, respectively. This study showed an acceptable CVI using the S-CVI/Ave because it exceeded 0.83 for relevancy and representativeness with an evaluation from eight experts [16]. This indicates that the translated versions of questionnaires are relevant and represent the domain. Based on the guidelines in article [10], the CVI value obtained acceptable validity. However, the CVI can be improved if the number of experts is increased to at least 10. In a validity study, there are two methods that can be used, either S-CVI/Ave or S-CVI/UA. S-CVI/Ave can also be used, since applying this method provides more convenience to achieve a better score compared to the conservative method of using S-CVI/UA.
In addition, this shows a very low value of S-CVI/AU [17]. This finding might be due to an expert’s agreement being too strict, based on the comments expressed by each expert who has participated in this study. This reason has also been identified and reported in Polit and Beck, [15]. Therefore, we also agreed with Polit and Beck, [15] that it is too conservative to claim 100% per cent agreement. This is because there are experts who are familiar with psychological instruments but lack expertise in content evaluation. There are some experts who think they need to know about the questions being evaluated, in order to give a better score. The possibility of a low S-CVI value is also due to an expert not understanding the task or having a biased view.
Experts also suggested to separate this research questions, since we are targeting two different populations. Accordingly, the results of improvement and separation were discussed with other panels in the validation team. Based on the results of the study and the comments expressed by all experts, they were evaluated and used to rectify the latest version before the final version is piloted with the target population.
Future suggestions for selecting experts in content validity assessment participation should consider their expertise and knowledge as psychometric instrument assessors. The results of this study will improve, if the number of experts is increased to 10. The pilot tests were conducted among smokers and non-smokers. This study found high internal consistency with a Cronbach’s alpha of more than 0.85. This revealed that these questionnaires showed high reliability. This study also faced challenges in finding smokers for the pilot testing, since it was conducted and recruited only among the university’s students. During the pilot testing, several e-poster and logo related to antismoking campaign have been attached to identify the familiarity of the programme. Perhaps, 64% of non-smoker, while 100% of smokers have never seen the logo of that particular campaign and 40 to 56% non-smokers and 50 to 66.7% have never seen the e-poster related to that campaign.
In conclusion, the adapted translated version’s showed acceptable validity and reliability. This questionnaire is ready to be used for the target population. However, this questionnaire can be retested to confirm the validity and reliability using other types of validity and reliability technique.
Acknowledgements
This study was funded by Universiti Teknologi MARA (UiTM) under the Geran Penyelidikan MyRA Lepasan PhD (600-RMC/GPM LPHD 5/3 (100/2022). This study was obtained approval from UiTM Ethical Commitee (REC/10/2022 (ST/MR/208). All authors declare no conflict of intrest.
Author Contribution Statement
NHR designed, studied, conducted, and wrote the manuscript, while ANMR contributed to the translation phase. All authors participated in the final approval of the manuscript.
References
- 1.Schiavo r. Health communication: From theory to practice. 2nd editio. San francisco: Jossey-bass; 2013. [Google Scholar]
- 2.Karlsson l. Advertising theories and models – how well can these be transferred from text into reality ? [phd thesis] University of halmstad; 2007. Available from: https://www.Diva-portal.Org/smash/get/diva2:238064/fulltext01.Pdf. [Google Scholar]
- 3.Nutbeam D. The evolving concept of health literacy. Soc Sci Med. 2008;67(12):2072–8. doi: 10.1016/j.socscimed.2008.09.050. [DOI] [PubMed] [Google Scholar]
- 4.Chang C. Increasing mental health literacy via narrative advertising. J Health Commun. 2008;13(1):37–55. doi: 10.1080/10810730701807027. [DOI] [PubMed] [Google Scholar]
- 5.Wakefield M, Flay B, Nichter M, Giovino G. Effects of anti-smoking advertising on youth smoking: A review. J Health Commun. 2003;8(3):229–47. doi: 10.1080/10810730305686. [DOI] [PubMed] [Google Scholar]
- 6.Elrod JK, Fortenberry JL Jr. Advertising in health and medicine: Using mass media to communicate with patients. BMC Health Serv Res. 2020;20(Suppl 1):818 . doi: 10.1186/s12913-020-05599-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Snyder LB. Health communication campaigns and their impact on behavior. J Nutr Educ Behav. 2007;39(2 Suppl):S32–40. doi: 10.1016/j.jneb.2006.09.004. [DOI] [PubMed] [Google Scholar]
- 8.An J, Kwak H, Qureshi HM, Weber I. Correction: Precision public health campaign: Delivering persuasive messages to relevant segments through targeted advertisements on social media. JMIR Form Res. 2021;5(10):e33922. doi: 10.2196/22313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Turnbull s. Absolute essentials of advertising. 1st ed. London: united kingdom: Routledge taylors and francis group; 2022; [Google Scholar]
- 10.Yousef M, Dietrich T, Rundle-Thiele S, Alhabash S. Emotional appeals effectiveness in enhancing charity digital advertisements. J Philanthr Mark. 2022:27. [Google Scholar]
- 11.Yusoff MSB. Abc of content validation and content validity index calculation. Educ Med J. 2019;11:49–54. [Google Scholar]
- 12.Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (pro) measures: Report of the ispor task force for translation and cultural adaptation. Value in health. 2005;8(2):94–104. doi: 10.1111/j.1524-4733.2005.04054.x. [DOI] [PubMed] [Google Scholar]
- 13.Kulkarni M, Attal G, Vasundekar V. Computing in Engineering and Technology Advances in Intelligent Systems and Computing. 2020. Evaluating effectiveness of amfi campaigns: A study based on aida model; pp. 747–61. [Google Scholar]
- 14.Polit DF, Beck CT, Owen SV. Is the cvi an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459–67. doi: 10.1002/nur.20199. [DOI] [PubMed] [Google Scholar]
- 15.Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489–97. doi: 10.1002/nur.20147. [DOI] [PubMed] [Google Scholar]
- 16.Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–5. [PubMed] [Google Scholar]
- 17.Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J Caring Sci. 2015;4(2):165–78. doi: 10.15171/jcs.2015.017. [DOI] [PMC free article] [PubMed] [Google Scholar]