Designing and Conducting Knowledge, Attitude, and Practice Surveys in Psychiatry: Practical Guidance

Abstract

Knowledge, attitude, and practice (KAP) surveys are popular in health care because they provide useful information and appear easy to design and execute. There are subtleties, however, in such surveys that early career researchers need to be aware of. This article does not provide a detailed review of the subject, nor does it address theory; rather, it provides practical guidance on matters such as identifying the need for the survey; defining the target population; preparing the questions that address knowledge, attitudes, and practice; preparing options for the answers to the items in the questionnaire; deciding how to score the instrument and analyze the results; and validating the instrument. Specific examples are presented to help readers understand and apply the guidance in various contexts.

Knowledge, attitude, and practice (KAP) surveys are popular in the health sciences. In the field of mental health, for example, such surveys may assess health-related beliefs and behaviors in the context of specific illnesses or specific treatments. KAP surveys burgeon when novel situations arise, as is evident during the current COVID-19 pandemic, which has spawned several KAP studies in the population at large as well as in selected subpopulations, including health care workers.^1–3 KAP studies do not require a large budget, usually address an easily accessible population, and appear simple and straightforward; so they are sometimes conducted by students and other researchers who may not have an adequate theoretical and practical grounding in matters such as classical test theory and item response theory, reliability and validity, factor analysis and Rasch analysis, and others, all of which are important to the development and use of research instruments.

Besides having a grounding in theory, students and researchers need to know the practicalities of the development of a KAP instrument. Such instruments have a title, a short introduction that includes instructions, a section that collects sociodemographic and other background information, and the main section that comprises the questions and/or statements to which responses about knowledge, attitudes, and practice are expected. Issues that also need to be considered relate to how the answers to the items in the questionnaire are to be recorded and how the answers may be scored. In this article, we limit ourselves to providing a brief background to KAP studies and offer simple guidance on practical matters related to the development of a KAP instrument. Our purpose is to help inexperienced but enthusiastic researchers gain basic insights in the field.

What Is a KAP Survey?

KAP surveys originated in the 1950s in the fields of family planning and population research. Also known as knowledge, attitude, behavior, and practice surveys, these are now widely accepted for the investigation of health-related behaviors and health-seeking practices. A KAP survey is meant to be a representative survey of a target population; it aims to elicit what is known (knowledge), believed (attitude), and done (practiced) in the context of the topic of interest. Information is collected using semistructured or (more usually) structured questionnaires that are self-administered or administered by interviewers; both qualitative and quantitative data may be collected.^{4, 5}

Why Do a KAP Survey?

KAP surveys are reasonably easy to design, conduct, analyze, and interpret. KAP surveys have therefore become popular, especially in the field of public health, where they help provide valuable information for resource allocation in, planning of, and implementation of public health programs. Important reasons for conducting KAP surveys are listed in Box 1.⁵

A KAP survey should ideally precede an awareness program or an intervention program. The results of the survey will provide the inputs needed for the design of an effective program, as well as the baseline data for the future evaluation of the success of the program. Thus, KAP surveys can be used to assess the baseline levels of awareness about mental health and mental-healthcare-seeking practices before designing and implementing educational or interventional programs in the population of interest. These surveys may then be repeated post-intervention to gauge the impact of the intervention. Interim KAP assessments may also be scheduled, if considered necessary, to determine whether the program is performing in accordance with expectations and to introduce midcourse corrections as appropriate. Such interim assessments can rescue those programs the failure of which would otherwise be discovered only after program completion.

It goes without saying that a KAP survey is necessary only if no surveys have previously been conducted in the population of interest, if there are knowledge gaps despite previous surveys, or if there is a specific need as listed in Box 1.

Box 1. Important Reasons for Conducting KAP Surveys.

• To identify the baseline knowledge, myths, misconceptions, attitudes, beliefs, and behaviors in relation to a specific health-related topic

• To understand, analyze, and communicate about topics or situations of interest in the field

• To provide information on needs, issues, and barriers related to the development of effective, locally relevant public health interventions

• To measure post-intervention changes, and thus, the effectiveness of intervention programs that were aimed at correcting and changing health-related knowledge, attitudes, behaviors, and practice

How Is a KAP Study Performed?

In the sections that follow, we outline the basic steps for the conduct of a KAP study. There are many steps, including identification of the topic of study, selection of the target population, preparation of the KAP questions, provision of options for answers, development of a scoring system for the instrument, and validation of the instrument. Each of these is considered in turn; our focus is on practical guidance rather than on a theoretical overview.

Topic Identification and Selection of the Target Population

Identification of the topic of study and selection of participant group(s) for the survey are the starting points for a KAP study. Consider the example of a researcher who wishes to study knowledge about and attitudes towards electroconvulsive therapy (ECT). For such a study to serve its purpose, the target population needs to be very specifically defined. The target population could be patients with depression or relatives of patients with a major mental illness, because these categories of individuals are key stakeholders in the processes involved in consenting for ECT. Or the target population could be nurses, psychologists, non-psychiatrist medical professionals, or even psychiatrists themselves, because these are the persons who may offer guidance about ECT to patients who require or are advised the treatment. It is obvious that the different groups listed here will differ in knowledge about ECT and attitudes towards the treatment because of differences in their professional backgrounds. The questionnaire will, therefore, need to be tailor-made for the target population.

A good research question addresses a felt need. The target population for a KAP survey, therefore, selects itself, because it is the population in which the need exists. There is no purpose in surveying a population in which no need exists. For example, there may not be much literature available on knowledge about and attitudes towards ECT among medical students, or among bartenders, for that matter. This does not mean that a KAP study should be conducted among either medical students or bartenders, because neither group is likely to influence behavior and practice in the field and because neither group is representative of the population at large. The general population is a legitimate population for study if an appropriate sampling method is feasible; if a narrow target group is desired for study, as already mentioned, nurses, psychologists, or nonpsychiatric medical professionals can be surveyed, because the findings of the survey would answer research questions, identify points of concern, and suggest areas for action.

Preparation of the Questions

The preparation of the KAP questionnaire involves framing the questions, creating options for answers, planning the scoring, and validating the instrument. The “questions” can be in the form of actual questions or in the form of statements; for convenience, in the rest of this article, the term “questions” or “items” will be used to include both contexts.

The process of framing the questions that assess knowledge, attitudes, and practice begins with defining the expected level of knowledge in the targeted population. The assessment of knowledge is relevant only insofar as it influences attitudes and practice. Thus, for example, the researcher who wishes to study knowledge about and attitudes towards ECT in the general population must first decide what facts about ECT the general population needs to be aware of in order to form attitudes that are based on science and not on false information. Alternately, the researcher who wishes to study knowledge about and practice of ECT among psychiatrists must first decide what psychiatrists need to know about ECT to run an ECT clinic with competence. The researcher must have experience and competence in the field for this purpose. It would also help if the researcher were to discuss the topic individually or in a focused group with, for example, experienced psychiatrists, experts on ECT, patients, and/or a small convenience sample of volunteers from the target population.

Testing the knowledge about ECT in members of the general population would require questions not just about facts but about possible myths and misconceptions related to efficacy and adverse effects; other aspects of knowledge, such as those related to the history of ECT, pre-ECT evaluation, ECT techniques, and so on, are hardly relevant because members of the general population would not be expected to be knowledgeable about these topics. A similar strategy of framing of population-specific questions applies to the assessment of attitudes, which are defined as a relatively enduring system of beliefs surrounding a subject, object, or concept that predisposes a person to respond in a preferential manner.⁶ Likewise, a similar strategy applies to the assessment of practice, or behaviors related to knowledge and attitudes in the field.

There is no need to have separate sections in the questionnaire for items related to knowledge, attitudes, and practice. This is because there is often an overlap between the items; for example, “ECT is an outdated treatment” reflects both a lack of knowledge and a negative attitude.

Questions must be framed with care. As examples, researchers must avoid questions that are too easy or those to which the answer is obvious. They must also avoid questions that are too difficult because the questionnaire is not an examination form. Other questions to avoid are those that may mean different things to different people, contain difficult words or concepts, use technical terms or colloquial phrases, are long and hard to understand, or address more than one issue in the same question or has double negatives. Care must be taken to avoid framing questions such that the ideal answer to all questions is “Yes”; respondents will then get into the mental set of ticking “Yes.” Care must also be taken to avoid questions the answers to which can be guessed from the content of previous or subsequent questions. Problems in these regards are often but not always picked up during the validation process, and so, researchers must be aware of these problems in advance.

Preparing Options for Answers

When creating items about knowledge, attitudes, and practice, the options for the answers must be created with caution. For example, for the statement “ECT produces brain damage,” options for response can be “True/Don’t know/False” or “Agree/Don’t know/Disagree”; the “Don’t know” option prevents respondents from being forced into selecting an option that they may not truly endorse. The “Don’t know” option also decreases the likelihood that a question will remain unanswered if the respondent is uncomfortable with both “Agree” and “Disagree” options.

When offering options for answers, care must be taken to avoid offering too extensive a range of options. Thus, for the statement “ECT produces brain damage,” it would be absurd to require respondents to select from among “Very strongly disagree, Strongly disagree, Disagree, Don’t know, Agree, Strongly agree, and Very strongly disagree.” Why absurd? Because, respondents have to think longer about their choices and it, therefore, takes longer to complete the questionnaire; because it can be hard for a person to accurately differentiate between “strongly” and “very strongly”; and because, to the researcher who is analyzing the results, it wouldn’t really matter whether the answer is “Agree,” “Strongly agree,” or “Very strongly agree.” Such options for answers, at least for knowledge questions, also create difficulties in scoring, as will be discussed in a later section.

When assessing attitudes, increasing response options, as described above, could additionally offer a false sense of detail (granularity) because there is no gold standard against which less granular and more granular can be validated. Furthermore, unless anchors are provided for each rating point, it will not be easy for respondents to justify a choice of 2 versus 3, or 4 versus 5, on a Likert scale. So, the choice of response could become arbitrary, and the test–retest reliability could drop. For example, those who respond “Agree” will probably respond “Agree” a day later, but if they respond “4” on one day, they may respond “5” the next day. (Readers may note that these views should be interpreted as recommendations and not as diktats.)

Care must be taken to avoid suggesting options that lead to answering by guesswork. Lastly, it may be useful to have a few open-ended questions that allow respondents to express thoughts related to matters that are not addressed in the questionnaire. Information obtained from open-ended questions can be qualitatively analyzed.

Scoring the Questionnaire

Logically, a KAP questionnaire should have three subscale scores and no total scale score. This is because knowledge, attitudes, and practice describe different constructs that cannot be summated into a unified construct, much as we do not summate depression, quality of life, and activities of daily living ratings into a unified construct. This means that the assessment of reliability, validity, and other psychometric properties of the instrument must be performed separately for each subscale.

Having said so, it must be recognized that items in the questionnaire could overlap across the subscales. For example, responding “Yes” to the item “ECT produces brain damage” reflects both absence of knowledge and a negative attitude; and responding “Yes” to the item “I would be unwilling to accept ECT if recommended for me” reflects both a negative attitude and the respondent’s practice.

Scoring is usually simple for the knowledge items. For example, a score of 1 can be given for each correct answer and 0 for each “Don’t know” or wrong answer. Here, it can be understood why it is usually not a good idea to have Agree, Strongly agree, and so on as options for answers; if Agree is scored as 1 and Strongly agree is scored as 2, would this mean than somebody who ticks Strongly agree for a correct answer knows twice as much about the item as some who ticked Agree?

When analyzing knowledge scores, if all items are approximately equal in difficulty, simple summation suffices to yield a total knowledge score. However, if some items are more difficult than others, then answering these items correctly should have greater value than answering the rest correctly; this is when more advanced statistical models, such as Rasch analysis, may need to be considered.^{7, 8}

Scoring of attitude items can likewise be simply done as, for example, assigning a score of 1 for a response that indicates a positive attitude and a score of 0 for any other response. Alternately, a negative attitude score can be generated. Summation would yield a total scale positive (or negative) attitude score.

Examining summated subscale scores is useful when comparing groups, or when comparing subscales before and after an educational intervention. Often, however, an examination of the pattern of response to individual items could be more important than an examination of total knowledge or total attitude scores. Thus, for example, it could be more important to know what percentage of patients or relatives think that ECT produces brain damage than to know what their average knowledge score is.

If the total score is computed, it can be interpreted in the context of the maximum possible score. So, if the mean knowledge score is 7 in a group of patients, and if the maximum possible knowledge score is 14, this means that the average patient correctly answers only half of the knowledge questions.

No cut-off should be assigned to these total scores. This is because nobody is perfect, and so who is to decide how much a patient, or member of the surveyed group, should know?

Scoring and analysis of responses to the practice subscale can be similarly decided, based upon what practices are being measured.

Validation of the Questionnaire

The KAP questionnaire needs to be validated after it is prepared. This is most commonly done through the processes of face and content validation. In the simplest of terms, face validation examines whether the instrument is likely to do what it is intended to do, and content validation examines whether the instrument includes all necessary items, avoids unnecessary items, and is generally appropriately framed and presented.

For validation, the questionnaire is circulated among experts (commonly, 3–5 experts), each of whom independently rates each item on the questionnaire as satisfactory or unsatisfactory on a validation form; if unsatisfactory, the expert states reasons for the judgment and offers suggestions for improvement. The researcher calculates the content validity index, revises the questionnaire based on the suggestions, and circulates the questionnaire, once again, for approval.

As an additional and useful component of validation, the researcher would be well advised to administer the questionnaire to a small number (e.g., 5–10) of volunteers in the target population to understand the problems that they face in completing the questionnaire and to receive their suggestions. Ideas for improvements (that emerge from this process) can be incorporated into the questionnaire.

Summary

Creating and executing a good KAP study is not as easy as it appears. Much thought must go into the selection of the target population, the items in the instrument, the creation of options for answers to items in the instrument, the decision on how the instrument will be scored, and the validation of the instrument. It is hoped that readers will find the practical suggestions in this article useful; those who wish to read about scale development in greater depth are referred to the articles listed in the section “Recommended Reading.”

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Recommended Reading

1. Kyriazos TA and Stalikas A. Applied psychometrics: The steps of scale development and standardization process. Psychology 2018; 9: 2531–2560.

2. Furr RM. Scale construction and psychometrics for social and personality psychology [Internet]. London: SAGE Publications Ltd, 2011 [cited Jul 8, 2020], http://methods.sagepub.com/book/scale-construction-and-psychometrics-for-social-and-personality-psychology (accessed July 28, 2020).

3. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, and Young SL. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Front Public Health 2018; 6: 149.