Development and Validation of Social-Cognitive Theory Based Oral Cancer Awareness Assessment Tool for Adolescents

Abstract

Introduction:

Creating oral cancer awareness among adolescents will bring change by modifying the risk factors responsible for oral cancer. Social cognitive theory (SCT) is one promising theory for developing oral cancer awareness programmes among school adolescents. However, data are limited on SCT-based intervention assessment in creating oral cancer awareness among rural community. To develop and validate a SCT-based Oral Cancer Awareness (SCT-OCA) assessment/survey tool for evaluation of intervention implementation.

Material and Methods:

A mixed method design encompassing both qualitative and quantitative was accomplished to develop and validate the assessment tool for rural setting.

Methods and Material:

Domains and items for SCT-OCA assessment tool for adolescents were selected using subject matter expert. A 21-item assessment tool was developed using three rounds of Delphi technique and validated using Lawshe method. The knowledge on oral cancer and its risk factors and key constructs of social-cognitive theory was selected as items of the tool. The final assessment tool was translated to regional language, which was used for evaluation of intervention implementation. Statistical analysis was conducted using SPSS, version 16. Descriptive statistics includes means, standard deviations and frequency. Validation using Lawshe, component factor analysis and Cronbach alpha coefficients were calculated.

Results:

The overall content validity ratio was agreeable, and 21 items were finally selected in assessment tool. The overall Cronbach’s alphas of 0.718 for survey was acceptable. The agreement was good for the domains of tool.

Conclusions:

This study developed the SCT-OCA assessment tool for intervention specifically designed for adolescents to measure oral cancer awareness.

INTRODUCTION

Global Youth Tobacco Survey (GYTS), 2009, data highlight that about 14.6% of adolescents are indulged in one or the other form of tobacco, the root cause of oral cancer.[1,2] These rates are of high concern for the current adolescents generation as they almost always initiate tobacco consumption during this period.[3] Oral cancer being a major public health problem, often associated with avoidable risk factors like tobacco. Till date, oral cancer has resulted in delayed diagnosis majorly and its prognosis depends on the stage at the time of diagnosis.[4] Lack of awareness on oral cancer among school adolescents hinders its early recognition. School environment is one of the crucial areas in this aspect. Adolescence period is involved in development all kind of behaviours and if neglected results into harmful habits.[5,6] Engaging adolescents in the right direction as change agents to create awareness on oral cancer, and its risk factors will be a major step in curtailing the habit.[7] Various school interventions conducted on oral health education have been successful in the past.[8,9]

There is increasing need for theory-driven intervention programmes to enrich adolescents’ awareness. Social cognitive theory (SCT) is one promising theory for developing school-based interventions,[10] which emphasises on development of behaviour through dynamic interaction between personal factors (beliefs, attitudes), their environmental factors and their behaviour.[11] SCT specifies on a set of core constructs (components) for translating knowledge into effective health practices. The core components needed in adopting and maintaining healthy habits are knowledge of health risks and health benefits, which creates a precondition for change and beliefs of self-efficacy to control health habits. Further, this theory has a social element, which debates that new information and behaviours are learnt through observational learning and integrate a large number of discrete ideas and concepts into an overall structure of understanding, predicting and changing human behaviour.[12]

The applicability of SCT in several health promotional research and school programmes has demonstrated positive outcomes[13,14] like reduction in tobacco usage, increased awareness on detrimental effects of tobacco and oral hygiene maintenance.[15–17] Even significant behavioural developments were observed among preadolescents and adolescents as well. Studies using SCT in cancer awareness and prevention of tobacco consumption among adolescents have shown effective results[18] but studies on oral cancer awareness are limited. Existing assessment tools on oral cancer education does not incorporate questions related to the constructs of the SCT. A valid assessment tool is needed for adolescents to assess the effect of SCT-based oral cancer education interventions. Therefore, to achieve this, the existing study was planned 1) to develop and validate a SCT-based Oral Cancer Awareness (SCT-OCA) assessment tool among adolescents focusing on knowledge plus self-efficacy; and 2) to assess its relationship on knowledge and self-efficacy of the SCT-OCA.

SUBJECTS AND METHODS

Ethical clearance was obtained from the Institutional Ethics Board prior to start of the study (FDS/EC/2014-16/PGST/30).

Study design: Development of assessment tool was done using mixed method and encompasses qualitative and quantitative methods. For this development, Delphi technique was used. Development incorporates questions, which measures knowledge and SCT key constructs—modelling (teacher and peers), goal setting, outcome expectations, self-regulation and self-efficacy. In qualitative method, domains and data were identified by the experts and then quantitatively rated the items to assess the consensus of the items, which was further validated.

The study was organised in two phases [Figure 1].

Figure 1.

Conceptual framework for development and validation

PHASE 1 DEVELOPMENT PHASE

Round one Delphi

Step 1: Key variables considered were decided after interviewing with experts:

Subject Matter Experts (SME) are experts active in academics, research, years of experience and area of oral cancer expertise were identified. 18 experts selected were professors, subject experts and PhD scholars, comprised of heterogeneous group, preliminarily approached via in-depth interviews. Five open-ended questions were asked to respond, by giving a maximum of five suggestions. Experts provided feedback on assessment tool containing main domains including their contents, adequacy and clarity, the design including a number of key constructs of SCT, outcomes to be measured, how to measure them, comprehensiveness of information being covered, practicality of use and so forth.

Based on review of the literature, the domains (variables) of interest were specified and number of decided initial items was listed to develop the SCT-OCA assessment tool. Specific needs of adolescents on oral cancer awareness were identified during field visit, aided in tool development. The tool draft copies were created with improved accuracy and clarity from collated responses, and the literature reviews by removing redundant, irrelevant and non-applicable items.

Step 2: Decision on information required

The tool was composed of two-specified domains (26 items) focused on knowledge (18 items) and self-efficacy (8). 18 items were based on oral cancer and its risk factors. Eight items were based on SCT key constructs: self-efficacy, self-regulation, observational learning/modelling, goal setting and reciprocal determinism. These domains are focused for two purposes: Firstly, each symbolises a conceptually distinct realm—knowledge (cognitive domain) and self-efficacy (affective domain), which later develops into behaviour (psychomotor domain).[19] Secondly, key constructs of SCT highlight relationship between environment, cognitive and behaviour.[10] From this perspective, an individual’s behaviour is both influenced by and is influencing a person’s personal (knowledge, coupled with self-efficacy) and the environment aspects (social support).

Step 3: Determining questions

It should be easily understandable and self-explanatory includes general oral health, causes and signs of oral cancer, treatment and myths. Knowledge domain includes causes, signs and symptoms of oral cancer. The correct response was awarded score of one and zero for the wrong answer. SCT-based questions were positively keyed and measured on three point rating scale; and hence, there was no necessity of reverse coding.

Step 4: Determining sequence of items

The items under each domain started with simple facts that were easy for participants to answer. Questions provided were sequenced from simple to complex, generic to specific and testing mainly knowledge and self-efficacy.

Round two Delphi

The collated and revised tool after round one Delphi was circulated to assess appropriateness, clarity and applicability. The same experts assessed whether all domains and items refer to relevant aspects of SCT constructs, appropriateness for target population being measured and also to add, remove or alter any items as considered appropriate. Experts were requested to rate each items using a five-point Likert scale where ‘1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant and 5 = extremely relevant’. The mean and level of agreement were calculated.

Round three Delphi

The result from round two was analysed. Experts were again asked to review the responses and reconsider their ratings linked with those of the panel as a whole and revise them if desired using same Likert scale. Delphi technique was aimed to achieve consensus among experts. Each item was measured with mean score and per cent agreement. A point of consensus was reached with a mean score of ≥4.0 and a per cent agreement of ≥70%. The assessment tool was subjected to validation process.

PHASE 2: PILOT TEST PHASE

Validation of assessment tool: The developed tool from Delphi technique (Phase 1) was subjected to content and construct validation, face validation and linguistic validation before intervention.

I. Content and construct validity:

C. H. Lawshe method (1975) was used, where each item in tool was graded by experts on basis of three item scale ‘essential’, ‘useful but not essential’ and ‘not necessary’. Less valid items will be eliminated. Content Validity Ratio (CVR) for every item was calculated using formula: CVR = [(E − (N/2))/(N/2)]. Where E = experts number rating each item as essential (i.e. highly relevant or moderately relevant) and N = total experts number.[20]

Method: The same 18 experts checked for relevance and were requested to rate each item. The rated copies were entered in SPSS, and CVR was calculated for each item. Mean CVR across items was calculated as an indicator to overall content validity.

Construct validity using Principal Component Analysis (PCA) was evaluated using data from 124 participants. Prior to the analysis, assumptions in the correlation matrix were confirmed. The Kaiser-Meyer-Olkin (KMO) measure and Bartlett’s test of sphericity were assessed for the sampling adequacy. Depending on the data factorability, which was determined by a KMO value of > 0.5 and significance of Bartlett’s test of sphericity value (p < 0.05), a principal component factor analysis using orthogonal rotation was performed. This provides an interpretable structure to explain variances among the factors that are independent from each other. Then Eigen value, scree plot and variance were analysed to determine number of factors. Factor analysis with 0.40 for factor loading was performed to extract scale components to determine the patterns of the correlations, which are distinct and reliable factors.

Reliability: The content validated tool was administered on 124 adolescents (~20% of sample size of the main intervention study, which is to assess the effectiveness of SCT-OCA intervention) to test the reliability and it took half an hour to complete the procedure. The indicators used for internal consistency are Cronbach’s alpha coefficient (a). The scale ≥0.7 is ideally acceptable value.

II. Face validity:

Face validity was done using two methods: interview/probe and bilingual. For interview method, tool was distributed to 124 school adolescents were asked to respond for appropriateness, comprehension and logicality of questions. The investigator had detailed discussion about each item, evaluating their understanding about questions and appropriateness to elicit accurate response. Along with this, even bilingual method is employed to bilingual experts and was asked to assess the tool. Based on these, necessary modifications were made in tool.

III. Linguistic validity:

Linguistic validation produces a target language version of assessment tool that is conceptually equivalent to the original version. Experts who were well versed in regional language (Kannada) were identified in public health dentistry and oral and maxillofacial surgery departments. Questions formulated in English were translated and back translated to Kannada to assure that target language version. It should be conceptually corresponding to original version. Linguistic validation procedure was conducted in five steps: 1. Conceptual definition, 2. Forward translation, 3. Backward translation, 4. Pilot testing and 5. Proof reading as shown in Figure 2.

Figure 2.

Steps adopted in Linguistic validation process of assessment tool

Data analysis

Statistical analysis was conducted using SPSS, version 16. SME’s evaluations of each item were considered to preserve the structure of the SCT model. The best scoring item of construct was retained. All constructs are considered, which ultimately increased self-efficacy. Descriptive statistics was computed to summarise Delphi results. It includes means, standard deviations and frequency. CVR for each item was calculated based on Lawshe method. Construct validity was assessed using Principal Component Analysis (PCA). Cronbach alpha coefficients were calculated to assess internal consistency of construct in the questionnaire. A value ≥ 0.7 was considered acceptable as Cronbach alpha coefficient.

RESULTS

Profile of Delphi experts

For Delphi technique, 18 experts were selected. Their distribution based on gender and discipline is shown in Tables 1 and 2. Among the participants, 55.6% were female, while the remaining was male (44.4%). The selected participants have academic service experience of more than five years. Majority, sixteen out of eighteen participants (88.9%) were professor position, expertise in oral cancer while two (11.1%) was from different profession, one is school teacher and the other child psychologist.

Table 1.

Profile of expert panellists

Delphi method
	Round 1 (n=18)	Round 2 (n=18)	Round 3 (n=18)
Gender n (%)
Males	8 (44.4)*	8 (44.4)	8 (44.4)
Females	10 (55.6)	10 (55.6)	10 (55.6)
Profession Background, n (%)
Dental teacher	16 (88.9)	16 (88.9)	16 (88.9)
School teacher	1 (5.56)	1 (5.56)	1 (5.56)
Child Psychologist	1 (5.56)	1 (5.56)	1 (5.56)

*Distribution of experts according to gender shows that 44.4% were males and 55.6% were females

Table 2.

Distribution of experts according to their discipline

Discipline	Number	Percentage
Oral and maxillofacial surgery	04	22.2
Oral pathology	03	16.7
Oral medicine and radiology	04	22.2
Pedodontics	03	16.7
Public health dentistry	02	11.1
Child psychologist	01	5.6
School teacher	01	5.6
Total	18	100

Round one Delphi

After round one Delphi, conglomerating literature review and experts response to open ended questions, tool was composed of two domains: Knowledge and Self-efficacy with 26 items.

Round two and three Delphi

At the end of Delphi technique, after rating of tool by experts, two domains were retained, while five items (9, 13, 18, 22 and 24) among 26 have disagreement. The mean and percentage agreement score for items ranged from 4.0 to 5.0 and 70% to 100%, respectively. Experts remarked importance of social-cognitive theory-based assessment tool as it evaluates the self-efficacy of the individual in decision-making and self-regulation task. Following feedback, questions were modified and finalised. The final tool had two domains with 21 items after removing irrelevant or non-applicable items to improve accuracy and clarity [Table 3].

Table 3.

Delphi Results

Round-2						Round-3
Question Number	SD	Median	Mean	Level consensus %	Level of disagreement %	Question Number	SD	Median	Mean	Level consensus %	Level of disagreement %
Q1	0.00	5.00	5.00	100.00	0.00	Q1	0.00	5.00	5.00	100.00	0.00
Q2	0.32	5.00	4.89	88.89	11.11	Q2	0.32	5.00	4.89	88.89	11.11
Q3	0.24	5.00	4.94	94.44	5.56	Q3	0.00	5.00	5.00	100.00	0.00
Q4	0.24	5.00	4.94	94.44	5.56	Q4	0.00	5.00	5.00	100.00	0.00
Q5	0.46	5.00	4.72	72.22	27.78	Q5	0.43	5.00	4.78	77.78	22.22
Q6	0.32	5.00	4.89	88.89	11.11	Q6	0.24	5.00	4.94	94.44	5.56
Q7	0.24	5.00	4.94	94.44	5.56	Q7	0.00	5.00	5.00	100.00	0.00
Q8	0.32	5.00	4.89	88.89	11.11	Q8	0.32	5.00	4.89	88.89	11.11
Q9	1.26	2.00	1.94	11.11	88.89	Q9	0.57	1.00	1.28	0.00	100.00
Q10	0.24	5.00	4.94	94.44	5.56	Q10	0.00	5.00	5.00	100.00	0.00
Q11	0.00	5.00	5.00	100.00	0.00	Q11	0.00	5.00	5.00	100.00	0.00
Q12	0.00	5.00	5.00	100.00	0.00	Q12	0.00	5.00	5.00	100.00	0.00
Q13	1.24	2.00	2.00	11.11	88.89	Q13	0.57	1.00	1.28	0.00	100.00
Q14	0.24	5.00	4.94	94.44	5.56	Q14	0.00	5.00	5.00	100.00	0.00
Q15	0.24	5.00	4.94	94.44	5.56	Q15	0.00	5.00	5.00	100.00	0.00
Q16	0.32	5.00	4.89	88.89	11.11	Q16	0.00	5.00	5.00	100.00	0.00
Q17	0.32	5.00	4.89	88.89	11.11	Q17	0.00	5.00	5.00	100.00	0.00
Q18	1.31	1.00	1.78	11.11	88.89	Q18	0.57	1.00	1.28	0.00	100.00
Q19	0.32	5.00	4.89	88.89	11.11	Q19	0.00	5.00	5.00	100.00	0.00
Q20	0.24	5.00	4.94	94.44	5.56	Q20	0.00	5.00	5.00	100.00	0.00
Q21	0.24	5.00	4.94	94.44	5.56	Q21	0.00	5.00	5.00	100.00	0.00
Q22	1.11	2.00	2.06	5.56	94.44	Q22	1.02	2.00	1.89	0.00	100.00
Q23	0.32	5.00	4.89	88.89	11.11	Q23	0.00	5.00	5.00	100.00	0.00
Q24	1.25	1.50	1.83	11.11	88.89	Q24	0.24	1.00	1.06	0.00	100.00
Q25	0.32	5.00	4.89	88.89	11.11	Q25	0.00	5.00	5.00	100.00	0.00
Q26	0.32	5.00	4.89	88.89	11.11	Q26	0.00	5.00	5.00	100.00	0.00

Content and construct validation results

Content validity: The CVR value is presented in Table 4. The finalised assessment tool was relevant and simple to the target population [Figure 3]. The overall Cronbach alpha (internal consistency) is 0.718. The internal consistency of the items of questionnaire score improved to 0.768 after deleting redundant five items and is adequate to perform factor analysis.

Table 4.

CVR ratio

Sl No.	Question items	CVR
QI	Oral cancer awareness	1
Q2	Identification	1
Q3	Smoking as cause	1
Q4	Alcohol as cause	0.7
Q5	Pan chewing as cause	1
Q6	Sharp tooth as cause	0.7
Q7	Hereditary factors as cause	1
Q8	Contagious nature	0.8
Q9	HPV as cause	0.3
Q10	Early detection and cure	1
Q11	Quick referral and cost	1
Q12	Oral cancer and spread	1
Q13	Unexplained bleeding as sign	0.5
Q14	Identify all causes	0.8
Q15	Signs and symptoms	0.8
Q16	Gender and oral cancer	0.8
Q17	Age and oral cancer	0.7
Q18	Parent rejection due to tobacco	0.6
Q19	Role model	0.7
Q20	Self-efficacy on recognition	1
Q21	Self-efficacy on identification of cause	0.8
Q22	Self-efficacy to convince smokers	0.3
Q23	Self-efficacy on additional information	0.8
Q24	Self-efficacy on referral to HCW	0.5
Q25	Self-efficacy on family and friends education	0.7
Q26	Self-efficacy on belief in early diagnosis	0.8

Figure 3.

Domains of assessment tool from Delphi method

Construct validity: KMO sampling adequacy was acceptable (0.600) and Bartlett’s test is also significant (3018.337 P value =0.0001). Factor analysis was done to remove the redundant variable, to reduce the number of variables into definite number of components and to obtain item-total statistics values [Table 5]. The scree plot showed seven-component solutions based on the Eigen values above one, and all the item’s extracted commonalities were above 0.7, which is acceptable to yield better results [Figure 4]. Further analyses were performed to confirm extracted factor solutions. In the rotated component matrix, a total of seven components are considered for factor loading. A factor loading of more than 0.70 will have a high impact on the variables and others need attention for the quality improvement.

Table 5.

Item-Total statistics

Item-Total Statistics
Question items	Scale Mean if Item Deleted	Scale Variance if Item Deleted	Corrected Item-Total Correlation	Cronbach’s Alpha if Item Deleted	n
Oral cancer awareness	18.57	41.271	0.232	0.765	124
Identification	19.05	40.859	0.309	0.760	124
Smoking as cause	18.56	41.208	0.244	0.764	124
Alcohol as cause	19.09	41.122	0.336	0.759	124
Pan chewing as cause	18.52	41.569	0.252	0.764	124
Sharp tooth as cause	18.52	39.373	0.283	0.764	124
Hereditary factors as cause	18.17	36.987	0.620	0.737	124
Contagious nature	18.31	40.868	0.343	0.759	124
Early detection and cure	18.48	38.544	0.484	0.748	124
Quick referral and cost	18.48	43.374	-0.014	0.782	124
Oral cancer and spread	18.58	39.579	0.489	0.750	124
Identify all causes	18.15	37.410	0.586	0.740	124
Signs and symptoms	18.31	40.816	0.346	0.759	124
Gender and oral cancer	18.44	39.452	0.389	0.755	124
Age and oral cancer	18.45	43.030	0.018	0.780	124
Role model	18.53	40.284	0.389	0.756	124
Self-efficacy on recognition	18.90	41.347	0.202	0.767	124
Self-efficacy on identification of cause	18.92	41.619	0.176	0.769	124
Self-efficacy on additional information	17.91	38.521	0.471	0.749	124
Self-efficacy on family and friends education	18.36	37.306	0.497	0.745	124
Self-efficacy on belief in early diagnosis	18.56	40.233	0.236	0.767	124

Figure 4.

Scree plot with Eigen values

Factor analysis of the 21 items yielded seven factors with Eigen values greater than one (4.505, 3.828, 2.805, 2.503, 1.898, 1.263 and 1.086), which explained 85.178% of the variance (21.455%, 18.230%, 13.356%, 11.917%, 9.039%, 6.012% and 5.170%), respectively [Figure 4]. A rotated solution was then arrived to simplify their interpretation and only loading factors higher than 0.4 were considered as significant based on the requirements of sample size [Table 6].

Table 6.

Item factor analysis results

Rotated Component Matrixa
Question items	Component
	1	2	3	4	5	6	7
Self-efficacy on identification of cause	0.876
Self-efficacy on belief in early diagnosis	0.868
Self-efficacy on recognition	0.851
Self-efficacy on family and friends education	0.795
Self-efficacy on additional information	0.516			0.406
Oral cancer awareness		0.900
Smoking as cause		0.888
Hereditary factors as cause		0.702
Identify all causes		0.693
Role model			0.916
Oral cancer and spread			0.904
Identification		0.423	0.602
Contagious nature				0.933
Signs and symptoms				0.932
Gender and oral cancer					0.879
Early detection and cure					0.865
Pan chewing as cause					0.681
Age and oral cancer						0.933
Quick referral and cost						0.927
Alcohol as cause							0.802
Sharp tooth as cause							0.791

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. ^aRotation converged in 7 iterations

DISCUSSION

This study designed a preliminary tool to assess knowledge and self-efficacy for SCT-OCA specifically for 11–16-year adolescents. Most available assessment tools measure oral cancer awareness is for adult population and not based on SCT construct. The available tool may not be able to accurately measure younger populations’ knowledge and self-efficacy. This may be due to their influence of culture or the context. Final tool with total of two domains and 21 items were developed using Delphi technique is a valid and reliable measure of SCT-OCA. Delphi technique is a structured consensus-building method encourages the expert to suggest honest opinion.[21] Initially, items for assessment tool were developed using in-depth interview method and justified through Delphi technique. Then the validation of assessment tool is essential to measure the domains developed as Delphi does not provide comprehensiveness. Hence, tool was subjected to psychometric analysis to assess validity and reliability. A methodical process was adopted to achieve internal validity. After finalising contents, tool was subjected to pilot test among 124 adolescents for construct validity, face validity and reliability. The reliability of questions was assessed on a group of 124 adolescents, and Cronbach’s alpha was 0.71 for questionnaire and showed that oral cancer awareness among this age group is low. Similar improved results are found in other studies conducted on different context of cancers.[22] The developed tool may be particularly relevant when adolescents are utilised as key change agents in diagnosing and monitoring the condition.

Adolescents are influenced by individuals, teachers, peer groups, environment factors, beliefs and attitudes.[23] Over the life course, adolescents develop strong reasoning skills. They cultivate logical and moral thinking, grow into more capable of abstract thinking and making rational judgements.[23,24] The prevalence of oral cancer in 15-year age group is 0.3% in India.[25] Tobacco and alcohol being major risk factors, in Karnataka, point prevalence of tobacco usage among 13–15-year-old was 4.9%,[2,26] which makes them an important target group for promoting oral cancer awareness through schools. Thus, questionnaire developed was based on relationships correlated to these circumstances.

The unique aspect of this assessment tool is based on social cognitive theory with self-efficacy component. Self-efficacy is one of the crucial constructs of the SCT, and inadequate research on oral cancer awareness addresses this topic. Hence, the self-efficacy construct will help the adolescents to build their skills in community engagement, which ripples in persistent promotion of oral cancer awareness.

CONCLUSION

The study developed the SCT-OCA assessment tool exclusively designed for adolescents to measure oral cancer awareness. The findings of current study offer evidence of validity and reliability of the SCT-OCA. It demonstrated the assessment tool is appropriate for epidemiological studies for adolescents in the evaluation of implementation effectiveness to improve knowledge and self-efficacy. The validated tool was based on SCT can be a valuable measure. Interventions targeted for adolescents will have the potential for a life-long impact on encouraging early diagnosis.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.