Crafting an effective questionnaire: An essential prerequisite of engaging surveys

Abstract

Questionnaires play a crucial role in biomedical research, enabling valuable data collection from individuals. However, the effectiveness of a questionnaire depends on its ability to engage respondents and gather accurate information. This manuscript delves into the intricacies of crafting an effective questionnaire, exploring the essential elements that contribute to their success and impact, and highlights the need for well-crafted questionnaires in biomedical research emphasizing the importance of maximizing respondent engagement, obtaining reliable data, enhancing data completeness, minimizing nonresponse rates and bias, and facilitating data analysis. The article further sheds light on the factors influencing questionnaire effectiveness, including clear and concise language, logical flow and structure, relevance and significance, avoidance of double-barreled and leading questions, and using balanced response options and skip questions. This narrative review examines how each factor contributes to questionnaire quality and offers examples to illustrate their importance. Moreover, it emphasizes the significance of investing time and effort in designing effective questionnaires to ensure the validity and success of biomedical research. By understanding the art of engaging surveys, researchers can optimize their data collection processes and enhance the reliability and validity of their findings.

INTRODUCTION

Designing a questionnaire is the most critical part of the research. Even though a questionnaire may appear as a simple list of questions to a young researcher, it aims to gather quantitative and qualitative information relevant to the research.[1] A carefully crafted questionnaire consists of questions framed to provide valuable insights and data by asking participants specific questions about their perspectives, experiences, or attitudes.[2]

Several factors determine the effectiveness of a questionnaire, commencing with the type and quality of questions asked to the art of engaging the study participants.[3] A well-crafted questionnaire can drive the participant’s mind to participate in the research, bring out accurate responses, and maximize the quality of the collected data.[4]

Although young researchers take reasonable care in crafting a questionnaire, right from considering the target group or population to carefully considering research objectives, still many of the research and surveys encounter high nonresponse rates and bias, which are challenging and distort the study’s findings.[5]

To address these issues, continued efforts are required to raise awareness among young researchers about the art of crafting an effective questionnaire. Hence, in this manuscript, we look into the details of crafting a well-designed questionnaire, exploring the essential elements contributing to their success and impact. This may help engage participants in surveys, optimize their data collection, and enhance the validity and reliability of their findings.

WHAT IS A QUESTIONNAIRE?

According to Merriam-Webster’s dictionary, a questionnaire is “a set of questions for obtaining statistically useful or personal information from individuals,”[6] while Collins Dictionary defines a questionnaire as “a written list of questions which are answered by a lot of people in order to provide information for a report or a survey.”[7]

ROLE OF VARIABLES IN DESIGNING A PRECISE QUESTIONNAIRE

Once the study’s objectives are finalized, the researcher has to plan the collection of data on measured characteristics (numerically or in terms of categories) referred to as variables. At the very outset, based on the thorough review of literature, researchers need to decide and define each and every variable to be included in the study. Accordingly, questions pertaining to sociodemographic, dependent, independent, and confounding variables should be arranged sequentially under appropriate subheads in the questionnaire.

Sociodemographic/universal variables help in the identification of the study subjects, prove the comparability of the groups, and are useful for procedures like matching. These variables also offer essential details about the respondent’s characteristics, help identify trends, and assist in drawing meaningful conclusion from the data.

Study variables, such as dependent and independent variables, make the core of the research question and hypothesis. For example, in a case–control study testing the hypothesis that obesity is associated with acute myocardial infarction (AMI), obesity is the independent variable, and occurrence of AMI is the dependent variable. The dependent variable or outcome variable is the effect and the independent variable is the suspected cause.[8]

It is imperative to obtain data on the confounding variables as they interfere with the association between dependent and independent variables. High quantity of fats in the diet can be a confounding variable in the above-mentioned case–control study. Diet with high quantity of fats is associated with obesity and is an independent risk factor for AMI. Hence, questions pertaining to the dietary habits of study subjects must be framed to avoid subsequent spurious association.

While designing questionnaire, due importance should be given to measures of time as well. Duration needs to be asked in the context of symptoms, signs, occurrence of complications, effect of any therapeutic interventions, survival, exposure, etc.

Thus, careful consideration for clear and well-defined variables contribute towards a logical flow in the questionnaire, enabling researchers to obtain relevant and precise data.[8]

NEED FOR A WELL-CRAFTED QUESTIONNAIRE IN BIOMEDICAL RESEARCH

Crafting an effective questionnaire is essential to ensure meaningful and accurate data collection. Here are a few reasons why it is crucial to make a detailed questionnaire:

Maximizing respondent engagement

A detailed questionnaire captures the interest and attention of respondents, increasing their willingness to participate. It should be done to encourage respondents to give thoughtful and honest answers. Embedded questions can help reduce survey fatigue and dropout rates, leading to higher response rates and more reliable data.[9,10]

Obtaining accurate and reliable data

Well-designed questionnaires that are easy to understand and navigate can help ensure the validity and reliability of the data collected. The well-designed questionnaire uses clear, concise language, avoids ambiguity, and includes appropriate responses. This helps reduce respondent confusion and improves response quality.[11]

Enhancing data completeness

A well-crafted questionnaire should include all relevant questions to gather detailed information on the research topic. Missing or incomplete data can limit the validity and usefulness of the results. Researchers can collect complete information about their research objectives by carefully structuring questionnaires and including all necessary questions.[12]

Minimizing nonresponse rate

Minimizing the nonresponse rate is crucial for obtaining accurate and representative data from surveys or research. Determining an adequate sample size to tackle the problem of nonresponse rate is critical for any research. After the minimum required sample size has been determined, it is necessary to make modifications to account for the potential nonresponse participants. For instance, if the researcher anticipates a high nonresponse rate in a survey, they should recruit additional participants equaling the expected nonresponse rate typically by 20%–30% in addition to the original sample size calculated to avoid underestimating the sample size.[13] A well-crafted questionnaire design attracts participants and motivates them to provide complete and comprehensive responses to tackle the problem of nonresponse bias.[14]

Minimizing nonresponse bias

Nonresponse bias, in simple terms, represents the potential bias or distortion that arises in the results of a survey or research when there is a difference in characteristics or opinions of nonrespondents versus those who respondents, thus making the study outcome unrepresentative of the target population if nonresponders systematically differ from respondents.[15]

Participants may deny to be part of the survey due to lack of interest, time, or other potential reasons. Now consider if nonrespondents differ significantly from those who responded regarding the topic of interest. This will likely result in the survey results not accurately representing the views or attributes of the target population.[5]

Thus, ensuring an appropriately crafted questionnaire can appeal to a wide range of participants and ensure inclusivity to prevent nonresponse bias.[16] To achieve this goal, one can also avoid offensive and sensitive questions using simple and appropriate language and making the survey accessible to a wide range of demographic participants.

Facilitating data analysis

A well-structured and organized questionnaire makes data analysis more efficient and effective. An effective questionnaire uses logical question sequencing, employs appropriate scales or measurement techniques, and provides clear instructions. This allows researchers to easily code, categorize, and analyze the collected data, leading to meaningful and interpretable results.[17]

TYPES OF QUESTIONS IN THE QUESTIONNAIRE

A questionnaire may consist of the following types of questions:

• Dichotomous question: Have two possible responses, i.e., yes/no or true/false

• Multiple-choice question: Have a list of responses from which one or multiple can be selected

• Likert scale question: Measure the degree of agreement or disagreement on a scale

• Closed questions: Provides specific data from predefined responses

• Open-ended question: Provides qualitative data, such as the perspective or feeling of respondents towards the research topic.

FACTORS DETERMINING THE EFFECTIVENESS OF QUESTIONNAIRES

An extensive literature review and consultation with experts

Creating an effective questionnaire necessitates a thorough literature search and expert assistance. Professionals offer insightful recommendations pertinent to their relevance and validity while scrutinizing prevalent scholarly material that equips researchers with a comprehensive understanding of current knowledge levels and areas that require further exploration within their chosen field. The meticulous evaluation of available resources aids in preventing replication and ensuring cultural appropriateness, reinforcing both the reliability and applicability of the questionnaire.[2]

Clear and concise language

Clear and concise language is crucial to ensure that respondents understand the intent and meaning of the question. It is important to avoid complex sentences, jargon, or ambiguous terms that may confuse or mislead respondents.[17,18,19]

Here is an example of a question with clear and concise language “Are the training materials provided easy to understand?”

In this example, the question is straightforward and uses concise language to ask respondents specific questions related to the evaluated topic (i.e., training materials) and the aspect of evaluation (i.e., whether they were easy to understand).

Logical flow and structure

Organize the questionnaire logically and coherently related to variables mentioned earlier. Start with introductory questions such as demographic information of the study participants, then progress to specific and detailed questions related to study variables.[20] Use headings and subheadings to group related questions together.

Let’s consider a participant’s satisfaction survey for a sanitation campaign. Example of poor logical flow and structure in a participant satisfaction survey regarding a sanitation campaign in India:

Question 1: How happy are you with the cleanliness of the campaign location?

Question 2: Have you found the campaign interesting, educational, and engaging?

Question 3: How old are you?

Question 4: How likely are you to participate in such sanitation campaigns in the future?

In this example, the questions lack a logical flow and coherent structure. The question “How old are you” is a demographic detail, so it should be in the beginning. Here is how it could be improved:

Question 1: Hold old are you?

Question 2: How happy are you with the cleanliness of the campaign location?

Question 3: Have you found the campaign interesting, educational, and engaging?

Question 4: How likely are you to participate in such sanitation campaigns in the future?

By reordering the questions, the demographic information (age) is collected first, followed by questions about satisfaction and engagement with the campaign. This ensures logical flow and structure of questions so that respondents can navigate the survey easily.

Relevance and significance

Ensure that questions asked are relevant to the research objectives and align with the study’s significance.[21]

For example, Dr. ABC’s topic of research is the impact of the “Clean Village Initiative” in improving sanitation services. If the question “To what extent has the ’Clean Village Initiative’ helped XYZ village’s rural residents’ access and use sanitary services?” If asked, then this question is relevant and significant for Dr. ABC’s research on the impact of the “Clean Village Initiative” in improving sanitation services. It addresses the critical aspect of toilet availability and usage, a crucial indicator of the campaign’s success in promoting proper sanitation practices in rural households.

Custom-made questions like the ones mentioned above capture relevant information necessary to evaluate the impact of a sanitation campaign in India. Such targeted questions help participants understand the purpose of the questionnaire to provide thoughtful and meaningful responses that can inform future strategies and interventions in the sanitation sector.

However, suppose a question such as “To what extent has the Pradhan Mantri Awas Yojana (PMAY) improved housing conditions and homeownership among rural residents in XYZ village?” is asked. In that case, it is irrelevant and insignificant for the “Clean Village Initiative” context. Inquiring about PMAY is no way contributes to the understanding or assessment of the sanitation campaign’s effectiveness and outcomes.

Avoid double-barreled questions

A double-barreled question presents two or more inquiries within a single sentence, making it difficult for the respondent to provide an accurate answer to each component separately.[22,23]

Here is a case of a double-barreled question: “Do you think that improving sanitation services and raising public awareness of proper waste management practices are equally important for the success of sanitation campaigns in rural areas?”

The availability of sanitary services and knowledge of waste management practices are two separate issues that are included in one question. It is advised to divide them into distinct and focused questions to prevent double-barreling and get precise answers for each question.

Avoid leading questions

A leading question is one that suggests or implies a particular answer or biases the respondent’s perspective. Such leading questions should be avoided as they unintentionally influence or bias respondents’ answers, leading to inaccurate or unreliable data. Instead, asking neutral and unbiased questions to gather objective information is important.[24,25]

Here is an example of a leading question “Don’t you agree that the Clean Village Initiative has been incredibly successful in transforming the sanitation conditions in rural areas of India?”

This leading question assumes the success of the Clean Village Initiative and biases the respondent toward agreeing with the statement, potentially influencing their response.

To avoid leading questions, it is better to ask open-ended, neutral questions allowing respondents to express their opinions or experiences. Here is a revised version of the question that avoids leading language “To what extent has the ’Clean Village Initiative’ influenced sanitation practices in rural India?”

The latter allows the respondent to provide their unbiased assessment of the impact of the Clean Village Initiative without leading them to a predetermined answer.

Avoid a negative question

A negatively framed question uses negative language leading to misinterpretation and is challenging to understand.[22]

Here is an example, “Isn’t it true that you haven’t observed any change in the sanitation facilities provided by the government in your village?”

This negatively worded question assumes the absence of any change and may bias the respondent toward a negative response.

Negative questions can be avoided by framing the question neutrally, such as: “Have you observed any change in the sanitation facilities provided by the government in your village?”

Use of statement instead of question

Statements in questionnaires can be less effective than questions, as they may lack clear guidance and direction. This ambiguity can leave room for interpretation and potentially make the item passive, which may reduce respondent engagement.[26]

When using a statement, it is important to frame a sentence that allows participants to express their agreement or disagreement. For example, “The sanitation campaign in my community has significantly improved hygiene practices.”

The participants may then be instructed to give a score on a scale of 1–10.

Use of few or too many response anchors

A response anchor refers to the labels or descriptors associated with response options provided to respondents. These help individuals understand and interpret the meaning of each response choice and provide a basis for selecting the most appropriate response.[26]

Response anchors are used frequently when Likert or rating scales are utilized to score or evaluate something based on a range of predefined response possibilities. Let us take an example of a survey assessing participant’s satisfaction; here response anchors that can be used are “very dissatisfied,” “dissatisfied,” “neutral,” “satisfied,” and “very satisfied.” This can facilitate participants to express their satisfaction level easily. It is important to carefully select and test response anchors to accurately capture the desired range of responses and for easy comprehension by the target audience.[27]

Few response anchors, such as a simple 2- or 3-point Likert scale, may restrict the range and granularity of responses. The lack of sufficient response options results in respondents feeling constrained to choose a response that does not accurately reflect their perspective. This may introduce response bias, and the data collected may be unreliable.[28]

On the other hand, numerous response anchors can overwhelm respondents and make the questionnaire complex and time-consuming. It may become challenging for individuals to differentiate between numerous closely spaced response options and can increase respondent fatigue, frustration, and reduced engagement, potentially resulting in incomplete or hasty responses, complicating data analysis, and increasing the risk of misinterpreting results.

The selection between a 5-point or 7-point Likert scale relies on the study’s specific requirements, objectives, and the survey participants’ characteristics. The 5-point scale is concise and clear, making it easy for researchers and participants to quickly understand each response option’s meaning. It provides adequate choices without overwhelming respondents. On the contrary, if the researcher desires distinct feedback or precise data, then the 7-point scale can help gather additional participants’ responses for expressing opinions. However, it is important to note that analyzing responses from a 7-point scale may require time and effort due to the increased complexity of the data. Both scales include a neutral midpoint, benefiting participants who genuinely feel indifferent toward a particular topic. Including a neutral option allows respondents to provide an accurate representation of their thoughts. However, some participants may use the neutral option as an easy way to avoid thoughtful responses. Ultimately, the choice of scale should align with your research goals and the characteristics of your survey population.[29]

The optimal number of response anchors depends on the nature of the construct being measured, the level of granularity desired, and the target audience. However, it is vital to pretest the questionnaire and consider respondent feedback to ensure that the chosen number of response anchors strikes the right balance between capturing meaningful distinctions and maintaining respondent engagement.

The researcher must pilot-test the questionnaire to ensure that response options are comprehensive and cover the range of possible answers.[30] Provide a balanced set of response options, including scales (e.g., Likert scale), multiple-choice options, or open-ended questions, depending on the nature of the survey and the type of data you want to collect.

Balanced response options prevent response bias, capture an array of opinions, facilitate standardization and comparability of data, enhance clarity for respondents, and simplify data analysis.[31]

USE OF PROPER SKIP QUESTIONS

Skip questions, also known as conditional or branching questions, allow respondents to bypass irrelevant questions based on their previous answers. These streamline the questionnaire and ensure that respondents only answer questions that apply to their circumstances, reducing unnecessary burden and enhancing the efficiency of the questionnaire.[32]

Have you ever been diagnosed with diabetes? (yes/no) (If “No,” please skip to question 5).

In this example, the skip questions allow respondents to bypass irrelevant questions based on previous answers.

Use of clear instructions

Authors should use simple explicit written instructions without ambiguous language that may lead to misinterpretation. Examples or additional explanations may act as a guide to respondents in providing accurate and meaningful responses.[21]

Concise and precise construct

Keep the questionnaire concise. Long and extensive questionnaires can overwhelm participants, leading to response fatigue or incomplete answers. Focus on collecting relevant information and avoid including unnecessary or redundant questions.[33,34]

Visual appeal and formatting

Visual cues such as headings, highlighting, clear fonts, appropriate spacing, colors, and well-organized sections improve readability and provide structure. These elements focus attention on key aspects, encourage active participation, and ensure high-quality data collection.[1,12,35]

Sensitivity and privacy considerations

If your questionnaire includes sensitive or personal questions, consider framing them respectfully and nonintrusively. Provide clear assurances regarding the confidentiality and anonymity of the participants’ responses.[36]

Here is an example of a sensitive question “Have you ever experienced violence at workplace?”

Workplace violence can be a sensitive topic for participants. Similarly, questions regarding a participant’s sexual orientation, political inclinations, and other sensitive issues can be perceived as intrusive, subjecting someone to distressing or traumatic situations. To respect privacy and minimize discomfort, it is vital to assure respondents of the confidentiality of their responses and emphasizing the purpose of the question is for research. Additionally, including an option for respondents to select “Prefer not to disclose” or “Not applicable” allows for privacy and inclusivity.[37]

PILOT TESTING

Prior to using a questionnaire, conduct a pilot test with a small sample of participants to identify potential issues and make necessary improvements before administering the questionnaire on a larger scale.[38]

This aids in the preliminary validation by identifying issues such as problems with language, wording, or structure that affect participant comprehension. At this stage, the appropriateness of the responses is also checked and corrected.[39]

VALIDATION AND RELIABILITY

Following a pilot test, content validation is carried out, with two or more subject-matter experts critically evaluating the questions to ensure they address the relevant subject matter in light of the study objectives.[40]

Construct validation is the next step, involving a comparison of responses from a larger population with established measures of the same concept.[41]

Finally, test–retest reliability and Cronbach’s alpha are used to assess the reliability of the questionnaire. Test–retest reliability evaluates consistency over time by administering the same questionnaire to the same group at different intervals, then correlating the responses. A higher correlation indicates greater consistency. Cronbach’s alpha, ranging from 0 to 1, assesses internal consistency, with values of 0.70, 0.80, and 0.90 generally considered acceptable, good, and excellent, respectively.[18]

ADAPTING OR USING COPYRIGHTED QUESTIONNAIRES

Researchers often include questions from questionnaires created by other authors or studies to fit their own needs, but these original questionnaires may be covered by copyright laws or licensing restrictions.[42,43]

The original questionnaires can only be reproduced, distributed, and used in accordance with the copyright or licensing rules by obtaining explicit permission from the original authors or copyright holders through formal written communication.[42,43]

Licensing rules, such as Creative Commons, must also be respected. Finally, all sources used must be properly acknowledged and credited, as failure to do so could lead to ethical and legal issues, and invalidate the research.[42,43]

Considering all the above-mentioned details the process of questionnaire development for any research is given in Figure 1.

Figure 1.

Process of questionnaire development

To conclude, here is the complete list of do’s and don’ts that every researcher must follow while crafting a questionnaire to ensure an engaging survey [Table 1].

Table 1.

Do’s and don’ts while designing a questionnaire for research

Do’s	Don’ts
Conduct a thorough literature review and consult experts before creating a questionnaire	Don’t skip the pilot testing of the questionnaire
Use clear and concise language that respondents can easily understand	Don’t use complex sentences, jargon, or ambiguous terms
Organize questions in logical flow (start with demographics, then progress to specific questions)	Don’t arrange questions randomly without proper structure
Include questions relevant to research objectives	Don’t include irrelevant questions that don’t align with research objectives
Use proper skip questions to bypass irrelevant sections based on previous answers	Don’t force respondents to answer questions that don’t apply to them
If using Likert Scales provide balanced response options or anchors (like 5-point or 7-point Likert Scales)	Don’t use too few or too many response anchors while using Likert Scales
Include clear instructions for completing the questionnaire	Don’t leave respondents confused about how to answer questions
Keep the questionnaire concise and focused	Don’t create lengthy questionnaires that cause response fatigue
Consider sensitivity and privacy in questions about personal matters	Don’t frame sensitive questions intrusively (if needed keep sensitive questions at last)
Use visual elements (headings, highlighting, clear fonts, spacing) for better readability	Don’t create cluttered or poorly formatted questionnaires
Keep only one concept per question	Don’t use double-barreled questions (two or more concepts per question)
Frame questions neutrally	Don’t use leading questions that suggest particular answers
Use positive language in questions	Don’t use negative questions that can cause misinterpretation
Obtain permission when using copyrighted questionnaires	Don’t reproduce copyrighted material without proper authorization
Validate the questionnaire through pilot testing, content validation, and construct validation	Don’t skip the validation process
Use questions to gather both quantitative and qualitative data as needed	Don’t restrict yourself to only one type of data collection

CONCLUSION

Crafting an effective questionnaire is crucial for efficient data collection, standardization of the process, and consistent interpretation. Additionally, it minimizes bias in participant responses, increases engagement and response rates, and captures complex or sensitive information. It also facilitates improved data analysis and interpretation, providing valuable insights for evidence-based decision-making in biomedical research.

Conflicts of interest

There are no conflicts of interest.

Funding Statement

Nil.