Designing A Questionnaire

In a survey, the researcher uses a questionnaire to gather information from the respondents to answer the research questions. A questionnaire is a very convenient way of collecting information from a large number of people within a period of time. Hence, the design of the questionnaire is of utmost importance to ensure accurate data is collected so that the results are interpretable and generalisable. A bad questionnaire renders the results uninterpretable, or worse, may lead to erroneous conclusions.

A survey can come in many forms: postal survey, telephone interviews, face-to-face interviews and internet surveys. Each type of survey requires a slightly different design. A self-administered questionnaire (e.g. postal survey) should have very clear instructions and questions, follow a logical order and avoid complex filtering. The respondents are more likely to answer truthfully without prompting from an interviewer. On the other hand, in an interviewer-administered questionnaire (e.g. face-to-face interview or telephone interview), the questions can be more complex as they can be clarified by the interviewers. However, the presence of an interviewer may “pressurise” the respondents to give “appropriate” rather than truthful answers.

WHAT IS A GOOD QUESTIONNAIRE?

A good questionnaire should be valid, reliable, clear, interesting and succinct.

Valid

A valid questionnaire should ask what it intends to ask, i.e. the questions should be phrased in such a way that the respondent understands the objective of the question. To achieve this, the questionnaire should be reviewed by the “content expert” during the pilot test (e.g. if the target respondent is a diabetic patient, then a diabetic patient should comment whether he understands the questionnaire). Any uncertainties and queries should be clarified till the question is clearly understood.

Reliable

A reliable questionnaire should yield the same answer if the same question is posed to the respondent repeatedly in a short span of time. This can be achieved by performing a “test-retest”, i.e. administer the same questionnaire to the respondent a second time and check for consistency of the answer. Any discrepancy in the answers could be due to lack of clarity of the questions and this should be reviewed and rephrased.

Interesting

An interesting questionnaire is more likely to be completed by the respondent and hence yields a better response rate. This requires the researcher to put some thoughts into asking questions that are relevant to the respondent and in a logical sequence.

Succinct

A succinct questionnaire asks questions that aim to answer only the research objectives. Any questions beyond the scope of the research should be excluded. It is common for researchers to “cast the net wider” so that they will collect more data, regardless of whether these data are important or not. This usually happens when the researcher has not properly thought through the research objectives. It runs the risk of asking too many questions and the questionnaire runs into many pages.

HOW TO DESIGN A GOOD QUESTIONNAIRE?

Developing a conceptual framework

The first step of designing of a good questionnaire is to construct a conceptual framework. The researcher needs to be very clear about his research questions and what “dependent” and “independent” factors he intends to investigate. Consider this research question: “What is the health-seeking behaviour of parents whose children have upper respiratory tract infection, and what are the associated factors?” I would develop a conceptual framework (Figure 1, Page 35) based on literature review, established theoretical framework and discussion with experts in that field. By creating the framework, the researcher can now ask questions regarding “parental health-seeking behaviour” (dependent variable) and associated factors e.g. education level, household income, age of child, etc. (independent variables). The importance of this framework is to ensure the research covers all relevant variables and any irrelevant variables can be excluded. This will answer the commonly asked questions: “Did I miss any important questions in the questionnaire?”, “Should I include/exclude this particular question?”

Figure 1.

Conceptual Model of Self Care (Simplified)

Asking the “right” questions

Now that you have developed the conceptual framework and you know exactly what questions you want to ask, it is time for you to design the questions in such a way that it is valid and reliable. The researchers have to brainstorm and come up with the preliminary questions.

Close- vs open-ended questions

These questions can be in the form of “close-ended” or “open-ended” questions (Table 1a). “Close-ended” questions provide options to the respondents and require them to choose one or more items from the list. “Open-ended” questions allow the respondent to express their opinions freely and they are not restricted by the options. The former is preferred if the range of answers are well known and the options are limited; the latter is preferred if the answer options are multiple and unknown. The answers to the open-ended questions require re-grouping before analysis.

Table 1. Asking the “right” questions.

1a
Closed-ended questions
Did you experience cough and cold in the past 6 months?
◻Yes	◻No
Think of the last time you had cough and cold. Did you consult a doctor?
◻Yes	◻No
Open-ended questions
What is/are the main reason(s) for you to consult a doctor for your cough and cold? (You can list down more than one reason)
___________
1b. Filtering
Think of the last time you had cough and cold. Did you consult a doctor?
◻Yes	◻No
If No, please go to Question 14.
1c. Sensitive questions
It is common for people to ask for antibiotics when they have cough and cold. Did you request for an antibiotic from the doctor for your cough and cold?
◻Yes	◻No
1d. Likert scale
We should always consult a doctor for cough and cold. (Please circle the answer)
1	2	3	4	5
Strongly disagree	Disagree	Neutral	Agree	Strongly agree
1e. “Double-barreled” questions
Did you experience sore throat and fever during your cough and cold?
◻Yes	◻No
Should be:
Did you experience sore throat during your cough and cold?
Did you experience fever during your cough and cold?
Or:
Did you experience the following symptoms during your cough and cold? (You can tick more than one box)
◻Sore throat	◻Cough	◻Fever
1f. Ambiguous Questions
How often do you visit the doctor for cough and cold?
◻Not at all	◻Not very often	◻Sometimes	◻Quite often
◻Very often
Change to:
How many times did you visit the doctor for cough and cold in the past 1 year?
_________ times in the past 1 year

Options/choices

The options available for each question should be as exhaustive as possible. This will ensure the respondent can find an option which best suits his answer. In order to determine the possible options, the researcher needs to brainstorm, review related published research, discuss with experts and if necessary, conduct a focus group discussion among the target respondents. To allow other possible options, the researcher can include an “Other: please specify ________” category as one of the options.

When assessing factual knowledge, it is important to include “Don’t know” as one of the responses as not all respondents may know the answer to the question. By not providing the option, the researcher is “forcing” the respondent to make a choice by guessing.

Filtering

In a questionnaire which has many parts, some of which need not be answered by the respondent, filtering is used to guide the respondent to answer only the relevant questions. (Table 1b) However, you should avoid using too much filtering as this may confuse the respondents and make the questionnaire complicated.

Order of questions

The order of the questions should flow in a logical sequence. Start with simple questions before moving to more complex questions. Some prefer to start with the socio-demography of the respondents while others will leave it to the last as it involves more personal questions such as household income, education level and religion. However, this depends on the how forth-coming the target population is. Sometimes, it helps by explaining to the respondent the reason for asking a personal question or by making a general statement to normalise the “sensitive” question. (Table 1c)

Likert scale

In questions which involve assessing attitudes or giving opinions, a scale with a range of responses is preferred to a yes/no answer. Likert scale (usually 5-point or 7-point) is a commonly used method. (Table 1d) It provides a measure of strength for a particular attitude or belief. It is possible to calculate mean scores for any given responses to statements (item scores)

Avoid double-barrelled questions

Another common mistake is asking a “double-barrelled” question. Avoid asking two things in one question (Table 1e) This will lead to difficulty in interpreting the responses when analysing the data.

Avoid ambiguous questions

Be as specific as possible when asking a question. For example, terms such as “frequent”, “always” and “often” may mean different things to different people. (Table 1f) Keep questionnaire items short, preferably less than 20 words. When scrutinising through the questions, ask yourself, “Is this question clear? Can it be more specific?”

Design the questionnaire with analysis in mind

When designing a questionnaire, it is crucial to pre-empt what kind of method will be used to analyse the data collected. Take for example, age. If the objective of asking the age is to find out the mean age of the participants, then an exact age should be captured (e.g. “What is your age? (at your last birthday): ______ years). On the other hand, if you are going to categorise them according to different age groups during the analysis, then you may want to structure the question according to different age categories (e.g. “What is your age? (at your last birthday): <18, 18-29, 30-50, etc.) If you are uncertain of what analysis you will be performing, it is always advisable to collect raw data, rather than to categorise them in the beginning. This will help to avoid problems with analysis after data collection and ensure that all data collected are relevant and usable. One practical way to do that is to draw up a “question-analysis” table in advance. (Table 2)

Table 2. “Question and Analysis”.

Questions	Analysis
Age	Mean (SD, range), ?categorise into groups
Ethnicity	%
Liker scale 1	% “agree” and “strongly agree”

Translation

A respondent should answer a questionnaire in a language which he or she is most proficient in. In a multi-lingual society like Malaysia, translating the questionnaire into different languages has become a “standard procedure”, especially for self-administered questionnaire involving the general population or patients. This is a crucial step because inaccurate translation of the questions or responses will result in collecting different information for the same question. This will lead to erroneous results and conclusions.

To avoid this, a “translate-back-translate” method is used. The researcher or a translator has to translate the questionnaire from English to Malay, and another independent person, who is unaware of the English questionnaire, will back-translate the Malay questionnaire to English again. The researchers (usually three or more people who are proficient in both languages) will then compare the original English questionnaire with the back-translated English questionnaire for any discrepancy, which may suggest inaccurate translation of the Malay questionnaire. These discrepancies will be discussed and the researchers will reach a consensus on the final translation.

Format

The final “touch-up” of the questionnaire is important because the “look” of the questionnaire may decide whether the respondent is going to fill it up. This is especially relevant for postal surveys. The title should be highlighted and it should reflect the main objective of the research. If possible, divide the questionnaire into sections according to the content (e.g. boxes with bold headings) and it should flow smoothly from one section to another with appropriate filtering. If your respondents involve older persons, bigger font size should be used. Finally, a cover letter stating the objective of your study, your affiliations, and, if appropriate, ensuring confidentiality and how you are going to use the information you have collected

Pilot test

Pilot test is a crucial step in the design of questionnaire before data collection begins. It will help to detect flaws in the questionnaire in terms of content, grammar and format. First, ask you colleagues, family or friends to comment on the questionnaire. This will pick up any mistakes in terms of content, grammar and format. This should be followed by asking the potential respondents to answer the questionnaire and provide their feedback. For those questions which you feel may be confusing or sensitive, it is important to ask the respondents to comment specifically during the pilot test.

CONCLUSIONS

A good questionnaire should be valid, reliable, clear, succinct and interesting. It is important to design the questionnaire based on a conceptual framework, scrutinise each question for relevance and clarity, and think of the analysis you are going to perform at the end of the day. A final touch-up will make a difference in the response rate and always pilot-test the questionnaire to perfect the questionnaire. Now you are ready to collect the data!