Development and Validation of Questionnaire to Assess Knowledge, Attitude and Practice of Frontline Health Workers on Identification of Oral Premalignant Disorders and Oral Cancer

Abstract

Introduction:

Understanding frontline health workers knowledge and attitude about different facets of oral cancer identification and management is important in development and evaluating interventions to promote timely cancer diagnosis yet there is a lack of validated, culturally relevant measurement tools. This study aimed to develop and validate a new questionnaire (HWOCI-Q) to assess KAP of frontline health workers on oral cancer identification.

Methodology:

The study was conducted in seven phases. In phase I, 46 English language items across five domains were generated through a literature search. In phase II and III, face and content validity were assessed using Cohen’s Kappa and the Content Validity index at both the item-level and scale-level by 13 subject matter experts. In phase IV and V final English questionnaire with 54 items were prepared and then rigorously translated into local Hindi. Finally, in phase VI and VII, pilot testing of the tool and reliability analysis using Cronbach’s alpha were conducted.

Results:

The new semi-structured, questionnaire called the Health Worker’s Oral Cancer Identification- Questionnaire (HWOCI-Q), comprised of 54 items under five domains. The final questionnaire had an overall Cohen’s Kappa of 0.93, an S-CVI/Ave of 0.94, and Cronbach’s α coefficient of 0.82 indicating excellent validity and reliability.

Conclusion:

This study indicates that the developed questionnaire is a valid and reliable instrument for assessing the KAP of frontline health workers on oral cancer identification. Thus, it represents a valuable tool that may contribute to more positive evidence-based research in future.

Introduction

Oral cancer presents a public health challenge with 389,846 new cases reported worldwide in 2022. As of 2022, India ranked first among the top ten countries with highest number of oral cancer mortality rates [1]. Oral cavity cancers are amenable to secondary prevention because they can be detected and addressed at precancerous stages through screening using cost effective techniques [2, 3]. National Health Mission, GoI [4] and National oral health program, MoHFW [5] also emphasized the need for oral cancer screening. Screening via conventional oral examination remains the first line approach in identification of OPMDs as the procedure requires less equipment, time and resources, making it a desirable option for large-scale implementation in resource constrained settings [6]. As recommended by WHO [7] and NP-NCD programme guidelines [8], one of the most effective options accessible for oral cancer screening especially in developing countries with scarcity of skilled healthcare professionals is involvement of frontline health workforce. According to Sankaranaryana et al, proper oral visual screening in high-risk individuals has the potential of preventing at least 37,000 oral cancer deaths worldwide.[2] Meta-analysis by Dahy Sulaminan et al and Keerthi Gurushanth et al highlights diagnostic accuracy, pooled sensitivity and specificity of oral screening by COE performed by a FHW to be 88.8% ,75% and 91.9%, 97% respectively [9, 10]. However, challenges at the primary and secondary levels, combined with exposure to environmental risk factors, inadequate knowledge and negative attitudes, complicate efforts to address this issue. As a result, early detection is inadequate, and referrals are delayed, leading to higher oral cancer morbidity and mortality. Understanding FHW’s knowledge, attitude, and practice (KAP) toward oral cancer is crucial for developing effective interventions and promoting behavioral change. This gap in understanding hampers developing effective oral cancer training programs for the health workers. It becomes evident, therefore, that a more tailored instrument is needed, thus justifying the development and validation of the questionnaire. This novel tool is intended to be comprehensive, designed specifically for the FHW’s, by capturing a broad range of factors related to oral cancer identification [11–16]. The aim of the present study was to develop, validate, translate and pilot test a new questionnaire (HWOCI-Q) to assess KAP of frontline health workers on oral cancer identification.

Materials and Methods

Study design & setting

This is a cross sectional study conducted between February 2024 and June 2024 at CDER, All India Institute of Medical Sciences, New Delhi.

Sampling

Purposive sampling.

Study procedure

The development and validation of HWOCI-Q [17–23] was carried out in the following phases as shown in the flowchart (Figure 1).

Figure 1.

Phases of questionnaire (HWOCI - Q) development

Phase I

Item generation via theoretical background and extensive literature review-To establish the theoretical background and create items to assess the knowledge, attitude and practice of frontline workers on identification of oral premalignant lesions and oral cancer, an extensive literature review was initially performed. Many studies and tools that cover a wider spectrum of knowledge, attitude and practice assessment relating to oral cancer were also reviewed [24–30]. Hence, the first draft of semi-structured HWOCI-Q consisted of 46 items in English language with various response scales like multiple choice options, 5-point Likert scale and questions with multiple responses was initially developed. The time to complete the questionnaire was 10-15 min. The draft was critically examined by the expert committee to confirm its content.

Phase II

Face validation of initial draft

Face validity

Face validity refers to researchers’ subjective assessments of the presentation and relevance of the measuring instrument as to whether the items in the instrument appear to be clear, relevant, reasonable and unambiguous. The procedure for face validity was carried out as follows [31–34].

Preparation of face validity form

Face validity form was prepared as a word document first and later it was changed into an online google form. [https://forms.gle/wAHLEUXDB9Eb9iZK7]. The google form had name of author, introduction, project title and aim, instructions to be followed, rate criteria, rating scale, comments/suggestions for each item. Expert’s name, qualification, years of experience and designation were included at the end of the google form. A seven-point criterion was used to assess the face validity: (i) the clarity and unambiguity of items (ii) appropriateness of difficulty level of instrument for the respondents (iii) correct spelling of difficult words (iv) appropriateness of grammar (v) appropriateness of font size and space (vi) adequacy of instruction on the instrument (vii) reasonableness of items in relation to the perceived purpose of the instrument. This 7-point criterion was based on a dichotomous scale with categorical option of “Yes” and “No” which indicated a favourable and unfavourable item respectively. A common additional criterion on “structure of the instrument in terms of construction and well-thought out format” with options “the instrument needs no improvement”; “the instrument needs partial improvement” and “the instrument needs significant improvement” was added at the end of all criteria’s.

Selection of expert panel

Based on evidence [32, 36, 37], it is suggested that greater than 7 experts having experience in topic/work/subject can give enough and good recommendations. The two main guidelines that was used in the selection of experts are as follows: (i) Academic experts who have experience of ≥10 years and more in public health, public health dentistry, oncology, community medicine and national health programmes. (ii) Educators and researchers with expertise and skills in domains and thematic concepts based on career experience. A total of 13 experts were selected for our study. Table 1 shows the number of selected expert panels, designation, areas of expertise and number of years of experience in the field.

Table 1.

Number of Selected Expert Panels, Designation, Areas of Expertise and Number of Years of Experience

Panel	Designation	Area of Expertise	Years of Experience
1	State Nodal Officer	Public Health Dentistry	14
2	Technical Consultant, NP-NCD, Ministry of Health and Family Welfare	Community Medicine	11
4	Medical Officer	Dentistry	17
5	Associate Professor	Community Medicine	10
6	Associate Professor	Public Health Dentistry	14
7	Associate Professor	Preventive Oncology	13
8	Scientist E [Medical]	Public Health	15
9	Public Health Consultant	Public Health	11
10	Additional Professor	Community Medicine	15
11	Additional Professor	Community Medicine	15
12	Professor	Community Medicine	18
13	Consultant	Public Health Dentistry	10

Distribution of face validity form

The google form was sent to the experts via their professional e-mails along with the hard copy of questionnaire attached. Even all details were stated in the google form, at the time of requesting, the experts were briefed about the process. Fifteen days were given to revert back. After fifteen days, a reminder e-mail was sent to those who didn’t respond. Of the 13 experts ,11 of them returned their responses resulting in a return rate of 84.6%.

Response compilation

The responses were returned to the investigators via email. After receiving the completed forms, responses were compiled as shown in Table 2.

Table 2.

Responses Given by Experts for Face Validation

Panel	Comments/Suggestions
1,4,6,7,9,10	Age and years of experience can be collected as an open question
3,4,5,11	Replace the words “metastasize”, “screening”, “oral mucosa” and “buccal mucosa” with simpler words such as “spread”, “early detection”, “oral cavity” and “inner surface of cheek”
1,10	Add options “irritation due to sharp tooth” and “betelnut/paan with lime” in Q.8
2,3,7,8	Delete Q.14,20 ,45
2,11	In Q 30, replace the option “perform biopsy” with “perform TCC” as FHW are not allowed to perform biopsy
1,4,9	Questions on “National Health programmes”, “OVE equipment’s” “different types of oral premalignant lesions” and “type of online training” can be added

Calculation of Cohen’s Kappa index: (CKI)

Responses from the expert panel was analyzed using the CKI. A value of Kappa equal to +1 implies perfect agreement between two or more raters, while that of -1 implies perfect disagreement. If Kappa assumes the value 0, then this implies that there is no relationship between the ratings of the two raters, and any agreement or disagreement is due to chance alone

Phase III

Content validation: [33, 35–40]

Content validity refers to “the extent to which the items in a questionnaire are representative of the entire theoretical construct the questionnaire is designed to assess”. The same procedure which was carried out for face validity was followed here. The experts were asked to rate the instrument items for relevancy and clarity to the construct underlying the study on a 4-point Likert scale. (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). They were specifically requested to provide suggestions (for revision or deletion) for each question which they would have scored low on Likert Scale. (1 or 2). The content validity google form can be accessed at https://forms.gle/zq5BUKNPGX88n5TR9.

Content validity was calculated using CVI, as this is one of most commonly used method. The detailed process of calculation is described in the results section.

Phase IV

Final HWOCI-Q preparation

The initial 46 items pool was validated by a thirteen-member expert panel as explained above. Each expert indicated his/her decision (to remove, keep, modify) for each item and made comments for the modified items. Items which were consistently judged to be removed were eliminated and modification was made to the modified items. Based on the comments of most of the experts ,8 new items were also added to the pool (Table 3). Hence, after item elimination, modification and addition, 54 items in English language were used for the final HWOCI-Q preparation which was again sent to experts for reiteration.

Table 3.

New Questions Added to Final Questionnaire

S.No	Questions
1	Which of the following National health programmes are you working for ?
2	What is the most prevalent form of tobacco that you are aware of?
4	What are the adverse health effects of tobacco consumption?
5	What was the mode of training?
6	Are you aware of the equipment’s used for oral visual examination ?
7	Are you aware of the screening and management protocol given by government for oral cancer?
8	What equipment’s are available for identification of oral cancer at the health centre?

Phase V

Process of translation of HWOCI-Q [41–44]

We followed the “WHO Translation protocol” with the following steps: (1) translation of English version into Hindi by a qualified bilingual translator, (2) The Hindi version was translated back to English and then reviewed by the panel of content experts in the second round to ensure that the translation did not change the meaning of the content (content equivalence), (3) both the Hindi and English versions were given to a qualified bilingual translator for linguistic validation and to identify the similarities and differences between the two versions (semantic equivalence). In this step, the discrepancies between the two versions were assessed, to ensure both versions are identical in content and meaning (conceptual equivalence) using the bilingual translation method. After this, the tool was pre-tested to achieve the final item wording of the questionnaire (cultural validation).

Phase VI

Pilot testing

The HWOCI-Q was pilot tested among 15 participants to estimate the reliability of questionnaire. These 15 participants were not included in the data collection procedure for the main study.

Phase VII

Reliability analysis [43, 45, 46]

The reliability of questionnaire is considered as the consistency of survey scores across raters or over time. For our study, the internal consistency via Cronbach’s alpha index was used for calculation of reliability, since the values can be estimated after only one test administration and therefore the problems associated with testing over multiple time periods can be avoided. The value of coefficient alpha can range from 0 to 1. As a rule of thumb, the higher the reliability value, the more reliable the measure. According to Nunnally and Bernstein, one should strive for reliability values of 0.70 or higher [47].

Data analysis

The data was analysed using IBM SPSS (Statistical Package for Social Sciences) version 25.

Results

Details of Final HWOCI-Q

The questionnaire [48] for the current study was initially drafted by the first author (HN) and was subsequently refined through collaboration with all coauthors, all of whom possessed previous knowledge regarding oral premalignant lesions and oral cancer identification. The final HWOCI-Q (https://forms.gle/sqUrQyJmpbutZ6fN7 ) had 54 questions under five domains as shown in Figure 2. There were two skip questions (Q.no 45 and Q.no 50)

Figure 2.

Domains, Content and Items in Final HWOCI -Q

(i) Development of “Demographic and workplace” domain: A total of 9 items were developed in the form of multiple-choice questions in this domain. Age and years of working in village/community/field responses were collected in an open -ended format.

(ii) Development of “Knowledge and Awareness” domain: A total of 10 items were developed in the form of multiple-choice questions under knowledge domain. About 8 items were added in the form of subdomain 2 on awareness about the oral premalignant lesions and oral cancer. Here the response options were “Yes”, “No” and “Don’t know”. One point was given to a correct answer and a zero point was given to an incorrect and “don’t know” answer. A higher total knowledge score indicates better knowledge on identification of OPMD’s and oral cancer.

(iii) Development of “Attitude” domain: Attitude domain consists of six items using five points Likert’s scale. The scoring systems of “Strongly Agree”, “Agree”, “Neutral”, “Disagree” and “Strongly Disagree” with numerical scores 5, 4, 3, 2 and 1 were given to each answer respectively which respondent can indicate their degree of agreement towards the statement given.

(iv) Development of “Practice and Training domain”: Sixteen items of this domain were hoped to correspond with the respondent’s practice towards identification of oral premalignant disorders and oral cancer. This domain was assessed using multiple choice, five-point Likert’s scale of “Always”, “Sometimes”, “Often”, “Rarely” and “Never” category and “Strongly Agree”, “Agree”, “Neutral”, “Disagree” and “Strongly Disagree” with numerical scores of 5, 4, 3, 2 and 1 given to each answer respectively. One question on attending any previous training was a skip question.

(v) Development of “Referral strategies” domain: A total of 5 items were developed in this domain with response options as multiple choice. First question in this section, on referring a patient for further treatment was a skip question.

Face validity

Validity values have been calculated by using CKI. Supplementary Table 1 shows the Kappa values for each individual question which ranges from 0.89 to 0.97. The overall Kappa value for the HWOCI-Q is 0.93. All responses from experts were taken and the researcher made improvements to the instrument.

Content validity [39, 42]

Validity values have been calculated by using Content validity index. There are two forms of CVI: I-CVI and S-CVI. The scale level content validity (S-CVIs) is computed by using two methods. The first method requires universal agreement among experts (S-CVI/UA). The second method uses the average of the item-level CVIs (S-CVI/Ave). Prior to the calculation of CVI, the relevance rating of 3 and 4 were recoded as 1 and the relevance rating of 1 and 2 were recoded as 0. The results of content validity analysis are provided in Supplementary Table 2. According to a previous study, for panel experts more than 5, items with I-CVI less than 0.78 should be revised or deleted. In our study, out of 46 items that were content validated, 43 items with I-CVI > 0.78 were retained and 3 items with I-CVI score of 0.72 were revised. The S-CVI/Ave score was 0.94 (scale-level content validity index/average) which suggests good content validity.

Reliability

Internal consistency was calculated using Cronbach’s alpha for 54 items. The Cronbach’s alpha value for the domains ranged from 0.49 to 0.83 whereas for the overall scale it was 0.82. Supplementary Table 3 show overall Cronbach’s alpha values, item–total correlation coefficients (correlation between each item in the domain and its total score), and alpha values when each item was excluded respectively.

Discussion

Oral cancer is a disease with known high-risk factors and an asymptomatic phase with identifiable clinical features referred to as OPMDs. The role of early diagnosis via screening in cancer survival is well documented in the literature but our understanding of various processes and factors influencing early diagnosis is limited [6, 49, 50] .Since, oral cancer mortality rates are higher in rural population, MoHFW, GoI ,in 2005 brought in ASHA workers with the intention of making the facilities more palatable, accessible, and readily available to them. Studies conducted in different parts of India had proved that frontline health workers such as ASHAs, ANMs, NCD nurses etc had a positive attitude and can contribute to prevention of oral diseases by spreading awareness and knowledge at the grassroot level in their communities [51–53]. The roles of these frontline health workers in implementation of cancer screening program at the primary level is listed in Table 4.

Table 4.

Role of Frontline Health Workers in Oral Cancer Screening

Frontline workers	Roles and responsibilities
ASHAs	(i) Create awareness among the population about breast, cervical and oral cancers and their early signs and symptoms, adverse effects of tobacco consumption and motivating the population to undergo cancer screening
	(ii) Collecting responses from people and filling out forms/oral health cards
ANMs/NCD nurse/Male health workers	(i) Performing Visual oral examination
	(ii) Collecting responses from people and filling out forms/oral health cards

To the best of our knowledge, this is the first of its kind study to explore and develop a new questionnaire to assess KAP of frontline health workers on oral cancer identification, which has not previously been studied in India. Several studies worldwide investigated oral cancer awareness, knowledge, practices, and attitudes among dentists [24–26, 30, 52]. Only few Indian studies highlighted good knowledge and attitude of health workers towards oral cancer screening and prevention but lack of clinical skills to identify and differentiate between OPMDs and oral cancer [52, 54–57]. Existing tools [27–29] used in this context have significant limitations. Primarily, they target oral cancer patients and often fall short of providing comprehensive coverage of various oral cancer dimensions such as self-screening methods, diagnosis, prevention practices, referral pathway thus leaving the practices and training of health workers on oral cancer identification inadequately addressed. In this context, a search was conducted among the published literatures to find the availability of pre-validated questionnaire. There was neither a gold standard instrument nor a pre-validated questionnaire available. The study considered to validate the questionnaire by drawing rigorous methodological protocols from benchmark literatures. The HWOCI-Q has been successfully translated to the Hindi language by following a model of a back-translation process. This systematic procedure was needed to ensure the quality of translation outcomes. The study results indicated that the HWOCI-Q have a good face (Cohen’s Kappa: 0.93) and content validity (S-CVI/Ave: 0.94) as well as a high degree of consistency and overall reliability (Cronbach’s Alpha: 0.82). Its multidimensional structure provides a more in-depth understanding of health worker’s competencies related to oral cancer identification, justifying the addition of new items. Despite its comprehensiveness, our pilot testing revealed that the HWOCI-Q can be completed within an average of 10–15 min, affirming its practicality and feasibility in real world settings. While our study underscores the strengths of the tool and its practicality, we acknowledge the possible challenges that could arise in real world settings. The tool’s use among diverse populations could necessitate the availability of translators or culturally adapted versions of the questionnaire. Furthermore, despite the reasonable completion time demonstrated in the pilot testing, some individuals might resist answering a relatively longer questionnaire. To mitigate this, clear communication of the tool’s importance in enhancing oral cancer awareness and prevention could be emphasized. The main limitation of this study was the limited sample size. In addition to it, there was a lack of diversity in the sample, therefore, there is a need for evaluation within more diverse populations. Moreover, it was not possible to engage in more sophisticated psychometric analyses to test for construct validity and test -retest reliability [58].

Policy and research implications

As cancer knowledge is a key element of cancer control programs, leading to both early diagnosis of cancer and risk reduction, we hope this HWOCI-Q would aid in collecting accurate and reliable data that could act as a forerunner to a nationwide study on knowledge, attitudes and practices hence guiding the stakeholders and policymakers for development of targeted oral cancer training interventions, educational campaigns, and awareness programs for frontline health workers. Further, more research is advocated to appraise the utility of this questionnaire in various other South-east Asian countries.

In conclusion, the present study concluded that HWOCI-Q is a valid and reliable tool to assess KAP of frontline health workers on oral cancer identification. This will be an indispensable tool for not only oral cancer assessment but can be tailor-made for all other cancers included under NCD’s like breast and cervical cancer. This attribution is because of HWOCI-Q’s comprehensive coverage of domains for frontline health workers however its usefulness for other cancers needs to be validated in future studies.

Author Contribution Statement

Huma Nawaz: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review and editing; Harsh Priya: Supervision, Project administration, Validation, Writing – review and editing; Bharathi. M. Purohit: Validation; Kavipriya Outtamane: Validation; Pallavi Shukla: Validation; Charan Kamal Kaur: Validation ; Harsavardhan Nayak: Validation; Stuti S. Bhargava: Validation ; Jitendra Kumar Meena: Validation ; Sujal Parkar: Validation ; Baridalyne Nongkynrih: Validation; Harshal Ramesh Salve: Validation; Rakesh Kumar: Validation; Shekhar Grover: Validation ; Nilima Nilima: Software, Formal analysis.