Rena is a family planning trainee interested in sexual dysfunction. She wishes to explore this in a sample of young women. In a pilot study, she finds that participants interpret the question “Did you get wet during sex?” differently. Some have answered in terms of arousal (which is what Rena wanted to find out about), while others thought wet referred to sweat or urinary incontinence. Each participant answered consistently, but what they thought they were talking about differed between participants. In the final questionnaire, Rena included a short explanation of what wet meant.

Phuong is leading a study on new parents’ experiences of maternity services. She invites people to participate, and notes the age, sex, ethnicity, and reasons given for refusing to join the study. Most commonly, participants refuse because they are too busy, not interested in the research, or dislike the subject matter of the questionnaire. By keeping tabs on participants who refuse, Phuong can check that no particular subgroup of people is being excluded, and she can later formally compare the characteristics of responders with non-responders. An example of the form used for collecting exclusion/refusal data in Phuong’s study is shown below:

1. Be as careful and accurate as possible when entering data into a database.

2. Take breaks. Fatigue causes mistakes.

3. Double check your questionnaire against the data you have entered. You can either do this with every questionnaire, or by picking a random sample and checking how accurately the data in the database matches the answers on the questionnaire.

4. If possible, get a colleague to work with you (eg read out answers from questionnaires to put into a database, or double-enter the data). Some statistical packages include a warning message or tone for double data entry.

5. Run statistical frequencies on all items and scan the results for obvious anomalies. Are any data missing? Are there numbers that don’t seem right (for example, on a questionnaire when participants can only give answers in the range of 1-5, are there any numbers outside that group in your database?). Go back to your coding sheets and check all anomalous data.

6. Create codes for missing data. This allows you to locate errors quickly. For example, if participants refused to answer, or couldn’t answer a question, or only answered part of the study, you can build in codes to account for this. Remember to make each code distinct, so they cannot be confused with numbers elsewhere in database.

7. For questionnaires that have sections that should add up, ensure the answers tally.

8. When answers are missing or don’t add up, take time to locate incorrect answers. Check your database regularly as you add data, so that when errors arise it doesn’t take too much work to find them.

9. Make a note each time you clean your data, and flag where you’ve got up to each time you work on your database. That way, when you find errors, you will have a more precise idea where to locate the problem.

See also www.uiowa.edu/~soc/datarespect/data_training_frm.html

Gavin had completed a questionnaire study on use of hormone replacement therapy in women and their partners. He had used a number of standardised measures alongside a questionnaire he had created himself. Having finished his research, he was uncertain about how to report his findings, so he included analysis on every question within 14 tables. He also added 10 bar charts and two pie charts for good measure. Since he wasn’t sure what to say about them he let the data speak for itself. Unfortunately it didn’t, but Gavin was lucky enough to have a supervisor who explained he only needed a small number of tables and graphs to illustrate what he had found from his study.

Method of delivery

Pros

Cons

Practical notes

By post

Participants are sent a copy of the questionnaire by post and asked to complete it and return it to the researcher.^w5

Quick and easy to distribute.

Relatively inexpensive.

They are not useful for the study of very personal issues (without first giving the participant the option of taking part in the study) and have a notoriously low response rate since you are relying on the goodwill and co-operation of individuals.

Remember to enclose a detailed introductory letter, a complete, contact address, and a stamped addressed envelope so the participant does not have to pay postage.

You may need to send reminder letters and questionnaires to slow/non-responders.

By telephone

The researcher calls participants and completes the questionnaire over the phone, with the researcher reading out the questions and recording the answers.^w6

Quick and easy to complete.

Relatively inexpensive.

Due to ethical constraints and sample bias these are used less within health research.

You cannot control for participant refusal, which is often high.

Not suitable for those with hearing problems.

Can become laborious if calling someone who is lonely and wants to talk.

Remember to contact participants by letter in advance of your call – and offer them a chance to opt-out of your study (and avoid your phone call).

Many ethics committees won’t permit a study where cold calling is the main design.

By email

Questionnaires are sent to participants via email for completion.^w7

Easy to design and send out.

Can keep track on who has responded and who hasn’t, and send reminders.

Only suitable for participants with email access, and who can download a questionnaire.

Can lead to confusion, where participants print out questionnaire and answer it by hand, rather than on the computer.

See telephone interview above.  Participants need an introductory email announcing the research and an opt-out option.  Follow data protection legislation, and check sending emails don't breach confidentiality.

By a website

The questionnaire is placed within a website and participants are directed to this and invited to complete it. ^w8

A simple questionnaire can be easily designed and placed within a website.  Since sites offer more space, it’s possible to have more opportunities for qualitative feedback using this measure.

Participants are only those with access to the Internet.   You may find they are a non-representative sample since they’ll have a special interest for visiting your site (e.g. your site is about testicular cancer and they have a particular view about it, or experience of illness).  It is difficult to stop the same person answering the questionnaire a number of times over.

Check your site regularly to ensure you can access the questionnaire and that there aren’t any ‘bugs’ in it. 

Encourage participants to report problems with accessing the questionnaire online.

Participant completion with researcher present

The researcher can answer questions the participants may have, but the participant answers the questions.^w2

The researcher is on-hand to offer support and explain any questions participants might not understand. 

They can also be sure that questionnaires are completed and collected.

Participants can inadvertently be ‘led’ by asking the researcher for advice on how to answer the questions.

Ensure your staff have training and support in how to deliver and code questionnaires and manage participants.¹⁶

Researcher Administered

The researcher asks the question and fills in the appropriate answers as directed by the participant.^w9

The researcher can be certain the questionnaires are fully and accurately completed, and collected.

The researcher may ‘lead’ participants by their tone of voice or phrasing of questions.

Participants may not understand what is required of them and not answer in a ‘standardised way’.

As above.

If you are using standardised measures researchers have to read these out in exactly the same order as they appear written in the questionnaire.

Type of response required from participant on the questionnaire

You can analyse this data using…

Binary or yes/no answers

c² (chi squared), Spearmans, Wilcoxon, Mann Whitney, Kruskal Wallis etc.

Rating or visual scales

Pearsons, t test, analysis of variance (ANOVA) etc.

Open-ended (free text) replies

Thematic content or discourse analysis.