Reaching beyond the white middle classes

Short abstract

Apparently simple questions can easily be misunderstood or cause offence in disadvantaged groups. But such problems can be avoided by careful design, piloting, and administration

Most published questionnaire research has been done on university students or in business or healthcare settings in Europe and north America.¹,² This bias leaves us with many unanswered questions about large and often disadvantaged sections of the population. In this article, we consider how to overcome the problems with accessing disempowered and socially excluded groups, cross cultural issues, and participants whose physical or mental health may interfere with their ability to complete a questionnaire. We also discuss the training and support of researcher staff. Researchers should also bear in mind the general principles of questionnaire design, administration, and piloting covered in the earlier papers in this series.³,⁴

Understanding and meaning

The meaning of a question may be obvious to you and your research staff, but this does not mean all participants will interpret it similarly. Ambiguous questions will lead to responses that do not accurately capture participants' views² or to them not bothering to respond.⁵

The greater the social distance between researcher and participant, the greater the risk of misunderstandings. A common problem is when researchers use abstract concepts but participants interpret these literally. For example, a questionnaire seeking to measure emotional wellbeing might include a question “Are you blue?” but some participants may interpret this as an inquiry about their physical health (with blue referring to skin colour or a mark on the body). Whenever you ask about an abstract concept, include a prompt or example, and take careful note of people's reactions during the pilot phase.

Phrases that researchers use routinely may not be familiar to participants or may have alternative (and even opposite) meanings in the real world. Most people understand the word homosexual, but heterosexual is less widely used and may be wrongly interpreted as synonymous with homosexual (the alternative of which, for some people, would be normal). A simple descriptor (such as men who have sex with men, instead of homosexual) will extend the accessibility of your instrument.

Questionnaire items often include unconscious assumptions about how people live. A small survey of elderly people's alcohol intake, for example, classified 5% of respondents as drinking excessively when questions were framed in terms of number of drinks per day. But when a question was included on tots of alcohol added to tea and coffee (which respondents generally did not count as a drink), the prevalence of excessive drinkers doubled to 10%.⁶

Figure 1.

Even something as simple as “how many days of the week do you watch television?” can be interpreted in several ways (does days of the week include weekends; does “you” mean just me, or me and my family; does having the television on in the background count?⁷). Implicit in the question format is a middle class notion that you are either watching or not watching television.

Complex routing instructions (if no, go to question 3; if yes, go to question 12) reduce motivation and may lead to indiscriminate box ticking. If participants do not follow the correct routing, additional statistical analysis may be required. For example, if someone ticks they are a non-smoker but then indicates they smoke 20 cigarettes a day, both data items would need to be removed from the analysis.

Disempowered participants

People should feel confident to answer questions and express feelings openly, without shame, embarrassment, or fear of retribution. Perceived threat, stigma, and social pressure can all pose serious barriers to achieving this goal, and such influences will be hidden from researchers if the instrument is piloted only on articulate respondents with nothing to hide.

One key principle is to assure privacy and non-threatening surroundings when completing the questionnaire and total anonymity when analysing the responses. Geographical location may influence outcomes. In one study, participants gave different answers if they were asked questions on their doorstep than they did when asked the same questions within their workplace.⁸ Ask participants where they would prefer to complete the questionnaire. At the very least, offer a quiet corner away from curious eyes.

Box 1: An unintended threat

Mr Bakoyan is a Kurdish refugee, seeking asylum in the United Kingdom. He has brought his young son with him to the doctor to act as a translator and is sitting in reception awaiting his appointment. A receptionist who is finding out what patients think about the surgery's new opening hours approaches him. Mr Bakoyan's son reads the questions and tells his father “They want to know where we live, how much you earn, and what religion and ethnic group you come from.” Mr Bakoyan becomes highly agitated, and Bathilda, the receptionist asks the practice manager Amal for help. Amal volunteers at a local refugee centre and is able to talk directly to Mr Bakoyan, who is frightened for two reasons. Persecution over his religion and ethnic group are partly the reason he has sought asylum, and he thought the questionnaire might be used to identify and deport him.

A common bias arising from lack of privacy occurs when friends, relatives, or others interject (see bmj.com). In such cases, researchers must use their initiative and social skills to gain space for the intended respondent—for example, ask the partner to bring refreshments. Even when the third party says nothing, this can still affect the way a participant responds.⁵

Helping participants to complete questionnaires

Questionnaires completed by researchers can be a legitimate approach if they are a planned part of the study protocol,⁹ but researchers can subtly influence responses by inflections of the voice, facial expressions, or gestures.¹⁰ For example, they may unconsciously hurry through questions they find uncomfortable or perceive as unimportant. A bored or tired researcher will convey a lack of enthusiasm, which might be interpreted as “it doesn't really matter which answer you choose.”

Changing the order or wording of questions could render the instrument invalid.⁹ But participants rarely work through self completed questionnaires in a linear fashion. They tend to refer to the instructions page several times and return to previous questions once they have completed later ones. If a researcher is helping participants to complete a standardised measure, sticking too closely to a standardised script can be counterproductive.¹⁰

Postal surveys are able to reach high numbers of participants but carry the risk of low response rates.⁴ Researcher administered questionnaires may be more labour intensive, but do lead to more responses, albeit in a smaller sample than that used in postal surveys.

Background demographic data

Researchers often see the demographics section of a questionnaire as the least threatening and easiest to design—after all, who could be offended when asked to complete their age, sex, marital status, occupation, socioeconomic status, postcode, or ethnicity? In fact, these items may be perceived as highly sensitive and upsetting (box 1). Table A on bmj.com lists some challenges associated with simple demographic data. Place demographic questions at the end to minimise the threat to participants,³ and give clear information about why these details are required, what they will be used for, and how this personal information will be protected.

Minority ethnic groups

Indigenous Western people are generally more familiar with rating scales and indicating preferences or choices than most other groups. That is not to say people from other groups cannot understand questionnaires, but the items may have different meanings to them,¹¹ and they may find the notion of mutually exclusive responses incomprehensible.

When friends, family members, or untrained healthcare staff act as translators for questionnaires, they may alter meaning through an attempt to clarify questions or protect participants or themselves. The use of children as translators for other family members in research should be avoided. Even when someone seems highly literate in English, their functional health literacy (the ability to understand and process health information) may be much lower, leading to under-reporting of symptoms by the poor and uneducated members of ethnic groups.¹² Professional translation and interpreting services are essential,¹³ but these do not necessarily ensure full and faithful translation of meaning,¹⁴ since direct translations of words may simply not exist (box 2).

Participants from minority ethnic groups may experience multiple jeopardy; they may also be poor, socially excluded, chronically sick, and illiterate in any language. They risk being excluded from research because of inability to speak or read the questionnaire in whatever language it is written, failure to comprehend rating scales or question routing, or misconceptions about the purpose of the study. There is also the issue of what someone asking questions represents. For example, some participants may find a female researcher asking questions to a man, or vice versa, unacceptable⁹,¹⁶; and for some refugees, demographic questions may be highly distressing (box 1). All these difficulties can be minimised by working closely with representatives of the community at design stage and by careful piloting.

Box 2: An unintended offence

In a study of Sylheti speaking British Bangladeshis, Trish's team (whose research team included a bilingual anthropologist) found the phrase “how much exercise do you take?” was interpreted by participants as “how often do you say your prayers?” Several participants found the question offensive. Not only was there no direct translation of the word exercise, but the word chosen by the researcher had a duplicate meaning in Sylheti: the physical up and down movement involved in Muslim prayers. A good Muslim is expected to pray five times daily, so a question that implied any choice in the matter held the implication, unanticipated even by the Muslim anthropologist, that they were less than devout.¹⁵

Box 3: Visit from the wee angel

Petra has been variously viewed by participants in her questionnaire studies as a nurse, social worker, young mother, youth worker, prostitute, doctor, physiotherapist, “that nice lady from the health authority” (when she was undertaking a study that was independent of the health authority), and, her personal favourite, “a wee angel—with wings.” In one study she worked on, participants were asked rate their satisfaction with a recent hospital stay.¹⁹ Many participants were delighted to have a visitor, and employed strategies to extend the conversation as long as possible. The question “during your stay in hospital were all your dietary requirements met?” rarely drew the required yes or no answer. Participants seized the opportunity to describe exactly what the food was like, what food they enjoyed eating, and how they'd like to see particular meals improved.

Disease specific issues

People who have a particular health problem or risk profile may hold beliefs or engage in behaviours that would be considered “abnormal” in the general population. For example, screening questionnaires for eating disorders might include questionnaire items such as “I always consider everything I eat very carefully,” or “I weigh and measure the portions of all my meals.” But a heightened awareness of food content and careful attention to portion size are considered key features of good self management of diabetes. Although several studies have shown a high incidence of eating disorders in young women with diabetes, a study that carefully took account of what is normal in people with diabetes showed no excess.¹⁷

Questionnaire interview as social interaction

A meeting between a researcher and a participant is a social interaction⁹,¹⁰,¹⁸ and often particularly important for people who are lonely, housebound, or socially excluded. As with other social research, participants rarely view the interviewer as a dispassionate scientist (box 3) and may erroneously associate them with the organisation that delivers care. These perceptions, and the quality of social interaction, undoubtedly influence responses.

Researcher training and support

The people who actually deliver questionnaires are often the least experienced members of the research team.²⁰ Even if they hold a relevant degree, they may not understand basic principles of recording, cleaning, and analysing data, the concept of standardisation, or the dangers to data quality of time pressures, embarrassment, burnout, and boredom (box 4).³,⁹

Personal questions are by definition difficult to ask, and whereas one person might recoil from asking about bodily functions, another might find questions about ethnicity, sexuality, or income uncomfortable. Supervisors must recognise these differences as legitimate and provide appropriate reassurance and support. Just because someone asks intimate questions in their clinical work does not mean they will find the same questions easy to ask in a research setting. Training researchers with the questionnaire they will be using is essential. The supervisor should note and deal with any problems the researcher has when asking particular questions in a real setting, and observe him or her administering the instrument, cleaning the data, and transferring responses on to a database.

An under-researched and often neglected issue in questionnaire research is the wellbeing and safety of the researchers. Participants may react to the invitation to participate with a somewhat unnerving appreciation of the warmth of human contact or with anger, arousal, frustration, or dismay. Informing the participant that they will have privacy without interruption can be misconstrued as a veiled sexual proposition. Professionalising cues (such as a name badge, letter of introduction, conventional dress code, and so on), a clear indication of the time allocated for the interview, and forewarning the participant that certain questions are of a personal or sensitive nature, can reduce such reactions.⁹ A list of help organisations or referral services should be available. Remember, though, that we can never fully predict the range or depth of human response to our research efforts.² The research supervisor should prepare researchers to expect a range of emotions (distress, arousal, embarrassment) in participants and to have specific strategies for dealing with these.

Conclusions

Questionnaire research needs to move beyond its traditional focus on the white middle classes. Working with disempowered, illiterate, socially excluded, and non-English speaking participants and asking questions about sensitive topics is hard work and time consuming and carries potential health risks for both participants and researchers. Researchers should be paid by the hour rather than by piecework and should take regular breaks from data collection to allow refreshment and reflection.²¹ Participants from particular vulnerable groups may need an interviewer who is similar to them. Study protocols should allow time for researchers to talk to the participant after the instrument has been completed. All these measures should be reflected in the project plan, timescale, and budget, and funders who are serious about redressing the gross bias in questionnaire research should be sensitive to the need for them.⁹

Box 4: The maverick researcher

Janine is a senior researcher, responsible for managing a project on teenagers' adherence to asthma medication. On several occasions she noticed Don, one of her assistants, filling in questionnaires at his desk. At first Janine assumed he was coding data, but finally caught him completing blank questionnaires. When challenged, Don replied “I have a really good relationships with the teenagers I interview. I remember all their replies and fill them in when I get back. I don't want to spoil our conversations by getting a questionnaire out.”

Summary points

Ignoring hard to reach groups in questionnaire studies biases data

Care must be taken to ensure questions are clear to everyone

How and where the questionnaire is administered and by whom can be important

Researchers need training and support to administer questionnaires effectively