Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2017 Apr 19;22(4):671–685. doi: 10.1111/bjhp.12239

What impact do questionnaire length and monetary incentives have on mailed health psychology survey response?

Kathryn A Robb 1,2,✉, Lauren Gatting 1, Jane Wardle 2
PMCID: PMC5655741  PMID: 28422369

Abstract

Objectives

Response rates to health‐related surveys are declining. This study tested two strategies to improve the response rate to a health psychology survey mailed through English general practices: (1) sending a shortened questionnaire and (2) offering a monetary incentive to return a completed questionnaire.

Design

Randomized controlled trial.

Methods

Adults (n = 4,241) aged 45–59 years, from four General Practices in South‐East England, were mailed a survey on attitudes towards bowel cancer screening. Using a 2 × 4 factorial design, participants were randomized to receive a ‘short’ (four A4 pages) or a ‘long’ (seven A4 pages) questionnaire, and one of four monetary incentives to return a completed questionnaire – (1) no monetary incentive, (2) £2.50 shop voucher, (3) £5.00 shop voucher, and (4) inclusion in a £250 shop voucher prize draw. Age, gender, and area‐level deprivation were obtained from the General Practices.

Results

The overall response rate was 41% (n = 1,589). Response to the ‘short’ questionnaire (42%) was not significantly different from the ‘long’ questionnaire (40%). The £2.50 incentive (43%) significantly improved response rates in univariate analyses, and remained significant after controlling for age, gender, area‐level deprivation, and questionnaire length. The £5.00 (42%) and £250 prize draw (41%) incentives had no significant impact on response rates compared to no incentive (38%).

Conclusions

A small monetary incentive (£2.50) may slightly increase response to a mailed health psychology survey. The length of the questionnaire (four pages vs. seven pages) did not influence response. Although frequently used, entry into a prize draw did not increase response. Achieving representative samples remains a challenge for health psychology.

Statement of contribution.

What is already known on this subject

  • Response rates to mailed questionnaires continue to decline, threatening the representativeness of data.

  • Prize draw incentives are frequently used but there is little evidence to support their efficacy.

  • Research on interactions between incentives, questionnaire length, and demographics is lacking.

What does this study add

  • Contrary to previous findings, questionnaire length did not influence response rate.

  • A £2.50 incentive increased response, while incentives of £5.00 and a £250 prize draw did not.

  • Achieving representative samples to questionnaires remains a challenge for health psychology.

Keywords: cancer screening, colorectal, incentives, length, mailed surveys, questionnaires, response rates, UK

Background

The majority of data gathered for health psychology research involve the use of questionnaires or surveys (Marks, Murray, Evans, & Estacio, 2015). However, the representativeness of the data generated from questionnaires is threatened by the persistent decline in response rates (Fulton, 2016; Tourangeau & Plewes, 2013). Low response rates have serious implications for the generalizability of the results of a survey and for the accuracy of any estimates made (Groves & Peytcheva, 2008). We know from previous work that lower socio‐economic groups are less likely to participate in research (McCaffery, Wardle, Nadel, & Atkin, 2002), and this will further undermine the generalizability of surveys. It is important that we identify methods to increase response, while also improving the distribution of responses, to be more representative of the population.

Edwards et al.'s (2009) Cochrane review identified that using monetary incentives nearly doubled response compared to no incentive, OR: 1.87 (CI 1.73–2.03), and many studies have shown that only a small amount of money (e.g., £5 or $1) is required to have a significant effect on response (Chan, Tse, Day, Tong, & Suen, 2003; Ulrich et al., 2005; VanGeest, Wynia, Cummins, & Wilson, 2001). Edwards et al.'s (2009) review also found that a bigger monetary incentive was more effective, OR = 1.26 (CI 1.14–1.39). However, the shape and strength of the relationship between incentive size and response is less clear, from the review with some studies demonstrating a positive correlation (Trussell & Lavrakas, 2004) and others not finding any significant correlation (Chan et al., 2003; VanGeest et al., 2001). Shortening the questionnaire length has also been shown to be effective in increasing response (Edwards et al., 2009; Singer & Ye, 2013).

However, much of the research into increasing mailed survey response has been conducted in the United States, with only a few studies carried out in the United Kingdom (Cartwright, 1986; Deehan, Templeton, Taylor, Drummond, & Strang, 1997; Kenyon et al., 2005; Roberts, Wilson, Roalfe, & Bridge, 2004). There are cultural differences between the United States and the United Kingdom; most notable for this research is that the United Kingdom has a national health service that is free at the point of delivery. Response to a survey sent within the UK health care context may therefore be quite different to those conducted in the United States. Also, none of the studies examined response to a health psychology questionnaire among a non‐patient population.

Many health surveys use prize draws as a cheaper way to incentivize participation as this requires less money than paying each participant individually (Bowling, 2014). However, most studies looking at the effectiveness of inclusion in a prize draw on response have found this type of incentive to be no more effective than receiving no incentive (Edwards et al., 2009). However, the effectiveness of prize draw incentives may be mediated by participant characteristics (Drummond, O'Leary, O'Neill, Burns, & Sharp, 2014; Spry et al., 1989).

Research on survey response has yet to build a clear picture on the interaction between demographic characteristics and the combined use of incentives and questionnaire length. There is variability in incentive effectiveness across different populations and demographic groups. For example, Boulianne (2013) found gender to interact with amount of incentive ($5 vs. $10) in increasing likelihood of responding to a web‐based survey about their general interests and hobbies, such that women were more likely to respond to $5 compared to $10. Other research has found a differential effect of incentive on response across levels of poverty. In a survey experiment using the US Survey of Income and Program Participation, Mack, Huggins, Keathley, and Sundukchi (1998) found $20, over no incentive, to be particularly effective in increasing response for participants who were living in greater poverty.

The present study

Achieving representative samples to questionnaires remains a challenge for health psychology but is under‐researched. Research into interaction effects between monetary incentives, questionnaire length, and participant demographic characteristics is lacking. Prize draws are frequently used but there is little evidence to support their efficacy. The current randomized trial was undertaken to compare the effect of four different incentive types and two questionnaire lengths on the response rates of a primary care population to a survey about bowel cancer screening. The aim was also to investigate any interaction effects between, questionnaire length, incentive type, and participant demographic characteristics.

The following hypotheses were made:

  1. A four‐page questionnaire will produce a higher response rate than a seven‐page questionnaire.

  2. A monetary incentive of £2.50 will produce a higher response rate than no incentive.

  3. A monetary incentive of £5.00 will produce a higher response rate than £2.50.

  4. A monetary prize draw incentive will produce a higher response rate than no incentive.

  5. No predictions were made about the interaction effect of demographic factors on the effectiveness of each of these administration techniques, due to the paucity of previous research in the area.

Method

Sample

Eligible participants were aged 45–59 years, registered with one of four General Practices in South‐East England (n = 4,583). This age group was selected because it is approaching the eligible age (60 years) for bowel screening in England. To ensure a socio‐economically and ethnically diverse sample, the four General Practices were selected based on their postcodes which were linked to area‐level deprivation from the Index of Multiple Deprivation (IMD; 2007) and ethnicity data from the 2001 Census. Frequency plots showed that the range of IMD scores within our data was broadly representative of 2007 IMD data for all of England (Communities and Local Government, 2007). General practitioners (GPs) were asked to exclude anyone known to have a recent diagnosis of cancer or those who were considered to be ‘vulnerable’ (e.g., learning disabled, cognitively impaired). This resulted in the exclusion of 342 (8%) people, leaving 4,241 potential participants, of which 1,996 (47%) were women. Participants were randomized by General Practice to one of eight groups (Figure 1).

Figure 1.

Figure 1

Open in a new tab

Study design.

Procedure

The remaining 4,241 potential participants were mailed a letter from their GP inviting them to participate in a ‘health and bowel cancer screening survey’, the study questionnaire, and a freepost reply envelope. Participants were randomized to receive one of two questionnaires; the ‘short’ questionnaire (four pages) was sent to 2,153 potential participants, and the ‘long’ questionnaire (seven pages) was sent to 2,088 participants. Participants were also randomized to one of four incentive groups as a ‘thank you for taking the time to fill out the questionnaire’ if they returned a completed questionnaire: (1) no incentive (n = 1084); (2) a £2.50 voucher (n = 1074); (3) a £5.00 voucher (n = 1057); or (4) the chance to win a £250 voucher in a prize draw (n = 1026). Participants were able to choose the shop their voucher could be used in from one of three popular shops in the United Kingdom – Boots, Tesco, or Argos. Table 1 describes the participant demographic characteristics across the eight experimental groups. Statistical tests confirmed that none of these demographics significantly differed across the experimental groups.

Table 1.

Participant demographic characteristics

Short questionnaire Long questionnaire Significance
No incentive £2.50 incentive £5.00 incentive £250 Prize draw incentive No incentive £2.50 incentive £5.00 incentive £250 Prize draw incentive
Age years Mean (SD) 51.3 (4.08) 51.2 (4.16) 51.0 (4.06) 51.0 (4.22) 50.8 (4.14) 51.2 (4.22) 50.7 (4.12) 51.3 (4.27) F (3872) = 1.61, p = .129
Gender
Male (n = 2245) 13.4 (269) 13.5 (270) 12.1 (243) 11.8 (236) 11.9 (239) 12.1 (243) 13.3 (267) 12.0 (240)
Female (n = 1996) 13.4 (250) 12.4 (231) 11.8 (220) 13.4 (250) 12.3 (229) 13.0 (243) 12.0 (223) 11.7 (219) χ2 (7, 3872)  = 5.20, p = .635
IMD 2007 Scores Mean (SD) 16.1 (13.5) 14.3 (13.8) 16.1 (15.1) 15.1 (15.0) 15.7 (14.5) 15.2 (14.5) 14.8 (13.6) 14.9 (14.1) F (3870)  = 1.00, p = .430
Open in a new tab

% (n) unless otherwise stated; SD = standard deviation.

Participants from the same household were sent the same version of the survey and incentive. Non‐responders were sent a reminder questionnaire after approximately 2 weeks. All letters included ‘return to sender if undelivered’ instructions on the reverse of the envelope. All questionnaires were mailed between June and December 2009. Ethical approval was granted by the NRES London Bridge Committee.

Measures

‘Short’ questionnaire

The short questionnaire contained four pages of questions. The questionnaire covered intentions to screen (two items), having previously done an FOB test (one item), six items from the State‐Trait Anxiety Inventory (STAI; Marteau & Bekker, 1992), seven items from the Behavioural Inhibition System (BIS; Carver & White, 1994), 24 items from the Rational‐Experiential Index (REI; Norris, Pacini, & Epstein, 1998), preference for information in the form of words compared to percentages (three items), demographic characteristics (age, gender, ethnic group, employment status, and living arrangements), experience of cancer themselves and among family and friends (three items), and whether they wanted to receive the results of the study and participate in future research (three items).

‘Long’ questionnaire

The long questionnaire contained seven pages of questions and included the same information and questions that were given in the short questionnaire. The long questionnaire also included questions on health and lifestyle (10 items), 14 items on coping strategies taken from Brief COPE inventory (Carver, 1997), and 16 items from the Miller Behavioural Style Scale (MBSS; Steptoe, 1989), and 21 items on attitudes towards bowel cancer screening adapted from several questionnaires (Lipkus & Klein, 2006; Miles, Voorwinden, Chapman, & Wardle, 2008; Tiro et al., 2005), and questions on perceived risk of bowel cancer (four items; Weinstein et al., 2007).

Gender data were also obtained from the General Practices to check that the intended recipient had completed the questionnaire. Socio‐economic deprivation was assessed by IMD (2007) score from participants’ home addresses.

Data analysis

SPSS Statistics for Windows, version 22.0, was used to analyse the data. Chi‐square tests were used to examine response rates between groups in terms of age (45–49 years; 50–54 years; 55–59 years), gender (female; male), IMD score 2007 quartiles, incentive (no incentive; £2.50; £5.00; £250 prize draw), and questionnaire length (short; long). Multiple‐variable logistic regressions were used to determine the independent influence on response rate of age, gender, IMD score, incentive group, and questionnaire length. Requests for access to the original data should be addressed to the corresponding author.

Results

Overall response

Of the 4,241 questionnaires mailed out, 185 (4%) were returned as undelivered. A further 184 (4.3%) of respondents were excluded from analysis because the self‐reported gender did not match the GP data, suggesting the intended recipient may not have completed the questionnaire. 156 questionnaires were returned blank and these were included as ‘no response’. There was a 41% response rate (n = 1,589/4,241–369).

Respondent characteristics

Older participants were more likely to return a questionnaire than younger participants, with those aged 55–59 more likely to respond (45.5%) compared to participants in the 45–49 (39.6%) and 50–54 (39.4%) age groups (Table 2). More women returned the questionnaire (45.9%) than men (36.5%). Participants from more affluent neighbourhoods were significantly more likely to return a completed questionnaire (IMD quartile 3 = 43.2%; IMD quartile 4 = 45.8%) compared to those from more deprived neighbourhoods (IMD quartile 1 = 39.1%; IMD quartile 2 = 35.9%).

Table 2.

Response rates by demographic characteristics, type of incentive, and questionnaire length

No response or returned blank % (n) (n = 2,283) Replied % (n) (n = 1,589) Significance
Age (years)
45–49 (n = 1,712) 60.4 (976) 39.6 (640)
50–54 (n = 1,318) 60.6 (771) 39.4 (501)
55–59 (n = 1,027) 54.5 (536) 45.5 (448) χ² (1, 3872) = 7.51, p = .006
Gender
Male (n = 2,148) 63.5 (1274) 36.5 (733)
Female (n = 1,909) 54.1 (1009) 45.9 (856) χ² (1, 3872) = 35.12, p < .001
IMD 2007 quartiles
1 (deprived) (n = 1,044) 60.9 (613) 39.1 (394)
2 (n = 987) 64.1 (606) 35.9 (340)
3 (n = 1,018) 54.0 (546) 43.2 (416)
4 (affluent) (n = 1,006) 53.8 (518) 45.8 (437) χ² (1, 3870) = 14.55, p < .001
Type of incentive
No incentive (n = 1,033) 61.8 (610) 38.2 (377)
£2.50 (n = 1,031) 56.7 (560) 43.3 (427)
£5.00 (n = 1,007) 58.0 (553) 42.0 (400)
£250 prize draw (n = 986) 59.3 (560) 40.7 (385) χ² (3, 3872) = 5.69, p = .128
Questionnaire length
Long (n = 1,985) 59.8 (1138) 40.2 (765)
Short (n = 2,072) 58.2 (1145) 41.8 (824) χ² (1, 3872) = 1.09, p = .297
Open in a new tab

IMD = Index of Multiple Deprivation; n = number.

Questionnaire length

Response rates did not differ between the ‘long’ questionnaire (40.2%) and the ‘short’ questionnaire (41.8%, Table 2). There were no significant differences in response to the different questionnaire lengths by gender or level of deprivation. However, participants aged 45–49 were more likely to respond to a shorter questionnaire, short = 42.9%,long = 36.4%; χ² (1, 1616) = 7.06, p = .008. The other age groups did not significantly differ in their response between the two questionnaire lengths.

Impact of incentives

Response rates did not differ significantly between people offered an incentive of £2.50 (43.3%), £5.00 (42%), entry into a £250 prize draw (40.7%), or no incentive (38.2%; Table 2). However, when the £2.50 and £5.00 monetary incentives were combined and compared to no monetary incentive, the group offered a monetary incentive had a higher response rate (42.6%) than those offered no incentive, 38.2%; χ² (1, 2927) = 5.31, p = .021.

There were no significant differences in response to the different incentives by age or gender. However, those who received a £5.00 incentive significantly differed in their response by neighbourhood deprivation, such that those from more affluent neighbourhoods were more likely to respond, IMD quartile 1 (deprived) = 39.3%, IMD quartile 2 = 35.4%, IMD quartile 3 = 45.3%, IMD quartile 4 (affluent) = 52.1%; χ² (1, 953)  = 6.00, p = .028.

Multiple‐variable logistic regressions

Logistic regressions were used to predict response rate (no response; response) using each predictor variable. In line with the univariable analyses, the multivariable analyses found a main effect of age, gender, and IMD score, with respondents more likely to be older, female, and from an affluent neighbourhood (Table 3). Questionnaire length was not related to response. Receiving a £2.50 incentive significantly increased the odds of returning a questionnaire compared to no incentive. Effect of incentive remained similar after controlling for age, gender, and IMD score, suggesting that there were no main interaction effects.

Table 3.

Logistic regression of response (0 = no response or blank; 1 = response)

Univariable logistic regression Multivariable logistic regression
Response OR (95% CI) Response OR (95% CI) Response OR (95% CI)
Age (years)
45–49 1.00 – 1.00
50–54 0.99 (0.85, 1.15) 0.98 (0.85, 1.14)
55–59 1.28 (1.09, 1.50)* 1.25 (1.06, 1.46)*
Gender
Male 1.00 – 1.00
Female 1.48 (1.29, 1.68)** 1.47 (1.29, 1.67)**
IMD 2007 quartiles
1 (deprived) 1.00 – 1.00
2 0.87 (0.73, 1.05) 0.86 (0.71, 1.03)
3 1.19 (0.99, 1.42) 1.16 (0.98, 1.39)
4 (affluent) 1.31 (1.10, 1.57)* 1.29 (1.07, 1.54)*
Incentive
No incentive 1.00 1.00 1.00
£2.50 1.23 (1.03, 1.48)* 1.24 (1.03, 1.48)* 1.21 (1.01, 1.46)*
£5.00 1.17 (0.98, 1.40) 1.17 (0.98, 1.41) 1.18 (0.98, 1.41)
£250 prize draw 1.11 (0.93, 1.34) 1.11 (0.93, 1.34) 1.08 (0.90, 1.30)
Questionnaire length
Long 1.00 1.00 1.00
Short 1.07 (0.94, 1.22) 1.07 (0.95, 1.22) 1.08 (0.95, 1.23)
Open in a new tab

IMD = Index of Multiple Deprivation; OR = odds ratio; CI = confidence interval.

*p < .05; **p < .001.

Discussion

Declining response rates are threatening the quality of data collected by health surveys. Previous meta‐analyses have identified small monetary incentives and shortening the length of questionnaires to be cost‐effective methods for increasing response rates (Edwards et al., 2009; Singer & Ye, 2013). However, research is yet to draw any clear conclusion about the combined influence of monetary incentives, questionnaire length, and individual differences on survey response.

Main findings

Contrary to previous findings, the current study found that questionnaire length did not influence response rates (Edwards et al., 2009). It is possible that the two lengths of questionnaire used in the current study (four pages vs. seven pages) did not differ enough to produce a significant difference in response. It would be useful to compare a variety of lengths to investigate the shape and strength of the relationship between questionnaire length and response.

The current study found that receiving a £2.50 incentive increased response compared to no incentive, supporting our first hypothesis and previous research showing that use of even a small monetary incentive in mailed surveys is more effective than not using an incentive (Edwards et al., 2009). Receiving a £5.00 incentive did not have a significant effect on response compared to no incentive, demonstrating that increase in incentive value and response do not share a simple positive linear relationship. The results may be usefully interpreted with reference to Social Exchange Theory (Dillman, Smyth, & Christian, 2009), which suggests that an individual's feeling of social obligation to complete a task is dampened when the amount of money given for completing that task approaches actual payment. In the present study, the offer of a £5.00 voucher may have been perceived as reaching a value equivalent to a payment for completing a four‐ to seven‐page questionnaire, thus reducing the intrinsic motivation of social obligation. Of note, people from more affluent neighbourhoods were significantly more likely to respond to the £5.00 incentive, while there was no difference in response for the £2.50 incentive across levels of deprivation. This suggests that different socio‐economic groups may have different perceptions about what constitutes a payment rather than an incentive.

A prize draw incentive was found to be no more effective than no incentive in increasing response regardless of length of questionnaire. This is in line with previous findings, with only one of 11 studies that looked at prize draw incentives in the Edwards et al.'s (2009) review finding a significant effect of prize draw incentives on response.

The only previous study simultaneously considering questionnaire length and prize draw incentive found prize draw incentive to be more effective with a shorter questionnaire (Spry et al., 1989). It would be useful to carry out further research in which survey length and size of prize draw incentive are experimentally manipulated to determine whether there is an interaction between these two factors.

In our study, older people, women, and those living in less deprived areas were most likely to respond, and these effects remained significant in multi variable analysis. The sample in this study was potentially limited in that it had only a small age range and used a measure of deprivation based on neighbourhood postcode that could be seen as less accurate than an individual measure (Fischbacher, 2014). However, Roberts et al. (2004) also found age, gender, and level of deprivation to influence response. In their sample, ages ranged from 18 to 100 years and deprivation was measured using the Townsend score. Affluent individuals and women are consistently found to be most likely to participate in surveys (Smith, 2008). This seriously impacts the generalizability of the results of any survey as they will not be representative of the general population (Groves & Peytcheva, 2008). It is imperative that future research into survey administration considers how to increase response from underrepresented groups based on demographic characteristics.

The findings from our research on survey response can also be considered within the broader context of incentivizing health behaviour. So far, research has found monetary incentives to be effective in improving health behaviours for changing small and discrete behaviours that are time‐limited (Lynagh, Sanson‐Fisher, & Bonevski, 2013). However, research has also shown that improving intrinsic motivations, such as altruism or self‐worth, can be more effective in changing behaviour than in increasing extrinsic motivation, such as monetary incentives (Sutherland, Leatherman & Christianson, 2008). Curry, Wagner, and Grothaus (1991) found that patients who received personal feedback (intrinsic motivator) alongside smoking cessation self‐help material were twice as likely to have abstained at 12 months compared to patients who had received a monetary incentive (extrinsic motivator), both with and without the personal feedback. People from lower socio‐economic backgrounds have been shown to be more responsive to monetary incentives particularly when the amount is perceived to address an economic barrier (Lagarde, Haines, & Palmer, 2007). It has been suggested that, under this circumstance, receiving money does not ‘crowed out’ the patients’ intrinsic motivations because the money is perceived as an enabler (Frey & Jegen, 2001). It could be argued that in the current study the £5.00 incentive ‘crowded out’ intrinsic motivation for people in the deprived group but not for people in the affluent group. Previous health behaviour change studies have found incentives higher than £5.00 to be effective in changing behaviour; for example, effective incentive sizes for smoking cessation trials have ranged from US$20 to US$750 (Volpp et al., 2006, 2009). Therefore, £5.00 is a comparatively low threshold for ‘crowding out’ to be experienced. It is likely that participants considered the task of completing and posting a survey to be undemanding and as having few economic barriers meaning payment for this task would equate to a much smaller amount.

Implications

This study contributes to an under‐researched topic in health research in the United Kingdom. Our findings suggest using a minimal amount of money (£2.50) to incentivize response may be justifiable to increase response rates without significantly impacting the demographic composition of the final sample. In contrast, our study suggests using a £5.00 incentive may disproportionately increase response rates of people from affluent neighbourhoods, leading to the further underrepresentation of people from deprived neighbourhoods in the sample.

Prize draws may seem an attractive option for increasing response rate for a mailed survey because it is cheaper than giving every respondent monetary payment for taking part. However, the results of this study support growing research evidence that this style of incentive is ineffective (Edwards et al., 2009). Young et al. (2015) have most recently looked at conditional versus unconditional incentives as a different way of conceptualizing pre‐payment and post‐payment, finding unconditional incentives to be much more effective.

Limitations

The topics of personal health practices, beliefs, and bowel cancer screening may have influenced response to this study (Cantor, O'Hare, & O'Connor, 2007; Van Kenhove, Wijnen, & De Wulf, 2002). For example, disgust is an emotion commonly reported in association with bowel screening that may have influenced response (Chambers, O'Carroll, Brownlee, Libby, & Steele, 2016; O'Carroll, Chambers, Brownlee, Libby, & Steele, 2015; O'Sullivan & Orbell, 2004). There is little to suggest that completing a health questionnaire with a portion of the questions related to bowel cancer screening will evoke a strong disgust reaction or that this will influence the effect of incentive or questionnaire length on response in a certain way. However, it is important to consider the topic of the survey when comparing the results to other studies.

It was unusual that the chi‐square test did not show a significant difference in response across incentive groups, while the univariable logistic regression did. This brings into question whether the analyses were appropriately powered. Post‐hoc power analyses determined that the chi‐square test was slightly underpowered (70.6%), while the logistic regression was above 80 (89.1%), which is considered high. This suggests that the £2.50 incentive significantly increased the odds of responding.

The age range of the sample was limited to 45–59 years. However, the findings remain important because surveys are becoming increasingly targeted to particular demographic characteristics, such as specific age groups. Further research could consider the impact of incentives on response for other age groups. From our study, we are unable to determine the extent to which non‐contact (i.e., the intended recipient does not receive the survey) accounted for non‐response. Non‐contact due to situational factors, such as the respondent having moved with no forwarding address, has been found to be higher for lower socio‐economic groups (Goodman & Gatward, 2008). This may lead to exaggerated results for non‐response in this group as these types of non‐response will not be affected by the experimental conditions. Similarly, there was no way of knowing whether intended recipients were the people sending back the surveys, although this was considered to some extent through comparing gender reported by participants with gender given by the GPs. This showed that at least 4% of responses were being completed by the wrong person.

Additionally, we did not investigate whether quality of response (i.e., how much effort participants put into completing the questionnaire) differed between the different methods of survey administration. Currently, there is no empirical way of determining the extent to which a response has been well considered, unless through self‐report.

Conclusion

The present study found that the length of the questionnaire (four pages vs. seven pages) did not influence response. An incentive of £2.50 slightly increased response but the offer of £5.00 or entry into a £250 prize draw did not influence response. Through looking at the interaction between demographic characteristics and incentive type, the current study was able to demonstrate that using monetary incentives does not necessarily improve the quality of mailed survey data, given that increase in response does not equate to a more representative sample or better quality of response.

The study found that a larger incentive of £5.00 significantly increased response if people were from an affluent neighbourhood but not if they were from deprived neighbourhoods. This suggests that monetary incentives of a certain amount could work to further marginalize people with lower socio‐economic status, which brings into question the appropriateness of using monetary incentives over other methods of increasing survey engagement. It is also clear from this result that, to identify ways of achieving greater and more representative survey uptake in the future, researchers need to consider what impact key qualities of their survey, and the ways in which it is being administered, are having on response. Investigating the potential interacting effect of survey properties and different demographic characteristics on response will also be essential in this endeavour. Achieving representative samples to questionnaires remains a challenge for health psychology research.

Conflict of interest

All authors declare no conflict of interest.

Acknowledgements

This article is dedicated in memory of Professor Jane Wardle (1950–2015). This work was supported by a Cancer Research UK Postdoctoral Fellowship to Dr Kathryn A. Robb (C9227/A8933) and a Cancer Research UK Programme Grant to Professor Jane Wardle (C1418/A414134). David Boniface's assistance with the randomization procedure is gratefully acknowledged. Thanks also to Jenny Taylor, Stephanie Schrempft, and Tomasz Nadarzynski for their assistance with data entry. The participation of the General Practices and respondents to the survey is gratefully acknowledged.

References

  1. Boulianne, S. (2013). Examining the gender effects of different incentive amounts in a web survey. Field Methods, 25(1), 91–104. doi:10.1177/1525822X12453113 [Google Scholar]
  2. Bowling, A. (2014) Research methods in health: Investigating health and health services (pp. 283–284). Maidenhead, UK: Open University Press. [Google Scholar]
  3. Cantor, D. , O'Hare, B. C. , & O'Connor, K. S. (2007) The Use of Monetary Incentives to Reduce Nonresponse in Random Digit Dial Telephone Surveys In Lepkowski J. M., Tucker C., Brick J. M., d Leeuw E. D., Japec L. & Sangster R. L. (Eds.), Advances in telephone survey methodology (pp. 471–498). Hoboken, NJ, USA: John Wiley. [Google Scholar]
  4. Cartwright, A. (1986). Some experiments with factors that might affect the response of mothers to a postal questionnaire. Statistics in Medicine, 5, 607–617. [DOI] [PubMed] [Google Scholar]
  5. Carver, C. S. (1997). You want to measure coping but your protocol's too long: Consider the Brief COPE. International Journal of Behavioral Medicine, 4, 92–100. doi:10.1207/s15327558ijbm0401_6 [DOI] [PubMed] [Google Scholar]
  6. Carver, C. S. , & White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS scales. Journal of Personality and Social Psychology, 67, 319–333. doi:10.1037/0022‐3514.67.2.319 [Google Scholar]
  7. Chambers, J. A. , O'Carroll, R. E. , Brownlee, L. , Libby, G. , & Steele, R. J. C. (2016). Colorectal cancer screening and perceived disgust: The importance of the “Ick” factor in faecal occult blood test uptake. Colorectal Cancer, 2(1), 6. doi:10.21767/2471‐9943.100013 [Google Scholar]
  8. Chan, T. M. F. , Tse, S. H. M. , Day, M. C. , Tong, E. T. F. , & Suen, L. K. P. (2003). Randomized trial of use of incentive to increase the response rate to a mailed survey. Asian Journal of Nursing Studies, 6, 36–43. [Google Scholar]
  9. Communities and Local Government . (2007). Indices of Multiple Deprivation 2007 [Data file]. Retrieved from http://webarchive.nationalarchives.gov.uk/20100410180038/http://communities.gov.uk/communities/neighbourhoodrenewal/deprivation/deprivation07/
  10. Curry, S. J. , Wagner, E. H. , & Grothaus, L. C. (1991). Evaluation of intrinsic and extrinsic motivation interventions with a self‐help smoking cessation program. Journal of Consulting and Clinical Psychology, 59, 318–324. doi:10.1037/0022‐006X.59.2.318 [DOI] [PubMed] [Google Scholar]
  11. Deehan, A. , Templeton, L. , Taylor, C. , Drummond, C. , & Strang, J. (1997). The effect of cash and other financial inducements on the response rate of general practitioners in a national postal study. British Journal of General Practice, 47, 87–90. [PMC free article] [PubMed] [Google Scholar]
  12. Dillman, D. A. , Smyth, J. D. , & Christian, L. M. (2009). Internet, Mail, and Mixed‐mode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley. [Google Scholar]
  13. Drummond, F. J. , O'Leary, E. , O'Neill, C. , Burns, R. , & Sharp, L. (2014). “Bird in the hand” cash was more effective than prize draws in increasing physician questionnaire response. Journal of Clinical Epidemiology, 67, 228–231. doi:10.1016/j.jclinepi.2013.08.016 [DOI] [PubMed] [Google Scholar]
  14. Edwards, P. J. , Roberts, I. , Clarke, M. J. , Diguiseppi, C. , Wentz, R. , Kwan, I. , … Pratap, S. (2009). Methods to increase response to postal and electronic questionnaires. Cochrane Database of Systematic Reviews(3), MR000008. doi:10.1002/14651858.MR000008.pub4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Fischbacher, C. (2014). Identifying “deprived individuals”: Are there better alternatives to the Scottish Index of Multiple Deprivation (SIMD) for socioeconomic targeting in individually based programmes addressing health inequalities in Scotland?. ISD Scotland/ScotPHO Retrieved from http://www.scotpho.org.uk/downloads/scotphoreports/scotpho140109-simd-identifyingdeprivedindividuals.pdf
  16. Frey, B. S. , & Jegen, R. (2001). Motivation crowding theory. Journal of Economic Surveys, 15(5), 589–611. doi:10.1111/1467‐6419.00150 [Google Scholar]
  17. Fulton, B. R. (2016). Organizations and survey research implementing response enhancing strategies and conducting nonresponse analyses. Sociological Methods & Research, doi:10.1177/0049124115626169 [Google Scholar]
  18. Goodman, A. , & Gatward, R. (2008). Who are we missing? Area deprivation and survey participation. European journal of epidemiology, 23, 379–387. doi:10.1007/s10654‐008‐9248‐0 [DOI] [PubMed] [Google Scholar]
  19. Groves, R. M. , & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias. Public Opinion Quarterly, 72, 167–189. doi:10.1093/poq/nfn011 [Google Scholar]
  20. Kenyon, S. , Pike, K. , Jones, D. , Taylor, D. , Salt, A. , Marlow, N. , & Brocklehurst, P. (2005). The effect of a monetary incentive on return of a postal health and development questionnaire: A randomised trial [ISRCTN53994660]. BMC Health Services Research, 5, 55. doi:10.1186/1472‐6963‐5‐55 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lagarde, M. , Haines, A. , & Palmer, N. (2007). Conditional cash transfers for improving uptake of health interventions in low and middle‐income countries: A systematic review. JAMA, 298(16), 1900–1910. doi:10.1001/jama.298.16.1900 [DOI] [PubMed] [Google Scholar]
  22. Lipkus, I. M. , & Klein, W. M. (2006). Effects of communicating social comparison information on risk perceptions for colorectal cancer. Journal of Health Communication, 11(4), 391–407. doi:10.1080/10810730600671870 [DOI] [PubMed] [Google Scholar]
  23. Lynagh, M. , Sanson‐Fisher, R. , & Bonevski, B. (2013). What's good for the goose is good for the gander. Guiding principles for the use of financial incentives in health behaviour change. International Journal of Behavioral Medicine, 20(1), 114–120. doi:10.1007/s12529‐011‐9202‐5 [DOI] [PubMed] [Google Scholar]
  24. Mack, S. , Huggins, V. , Keathley, D. , & Sundukchi, M. (1998). “Do Monetary Incentives Improve Response Rates in the Survey of Income and Programme Participation?” Proceedings of the Section on Survey Methodology, American Statistical Association, 529‐34. cited in Sutherland, K., Leatherman, S., & Christianson, J. (2008). Paying the patient: does it work? A review of patient‐targeted incentives. London, UK: QQUIP. Quest of Quality and Improved Performance. The Health Foundation.
  25. Marks, D. , Murray, M. , Evans, B. , & Estacio, E. V. (2015). Health psychology: Theory, research and practice (4th ed.). London, UK: Sage. [Google Scholar]
  26. Marteau, T. M. , & Bekker, H. (1992). The development of a six‐item short‐form of the state scale of the Spielberger State‐Trait Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31(3), 301–306. doi:10.1111/j.2044‐8260.1992.tb00997.x [DOI] [PubMed] [Google Scholar]
  27. McCaffery, K. , Wardle, J. , Nadel, M. , & Atkin, W. (2002). Socioeconomic variation in participation in colorectal cancer screening. Journal of Medical Screening, 9, 104–108. doi:10.1136/jms.9.3.104 [DOI] [PubMed] [Google Scholar]
  28. Miles, A. , Voorwinden, S. , Chapman, S. , & Wardle, J. (2008). Psychologic predictors of cancer information avoidance among older adults: The role of cancer fear and fatalism. Cancer Epidemiology, Biomarkers & Prevention, 17, 1872. doi:10.1158/1055‐9965.EPI‐08‐0074 [DOI] [PubMed] [Google Scholar]
  29. Norris, P. , Pacini, R. , & Epstein, S. (1998). The Rational‐Experiential Inventory, short form. Unpublished inventory. University of Massachusetts at Amherst.
  30. O'Carroll, R. , Chambers, J. , Brownlee, L. , Libby, G. , & Steele, R. (2015). Anticipated regret to increase uptake of colorectal cancer screening (ARTICS): A randomised controlled trial. Social Science & Medicine, 142, 118–127. doi:10.1016/j.socscimed.2015.07.026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. O'Sullivan, I. , & Orbell, S. (2004). Self‐sampling in screening to reduce mortality from colorectal cancer: A qualitative exploration of the decision to complete a faecal occult blood test (FOBT). Journal of Medical Screening, 11(1), 16–22. doi:10.1177/096914130301100105 [DOI] [PubMed] [Google Scholar]
  32. Roberts, L. , Wilson, S. , Roalfe, A. , & Bridge, P. (2004). A randomised controlled trial to determine the effect on response of including a lottery incentive in health surveys [ISRCTN32203485]. Bmc Health Services Research, 4(1), 30. doi:10.1186/1472‐6963‐4‐30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Singer, E. , & Ye, C. (2013). The use and effects of incentives in surveys. Annals of the American Academy of Political and Social Science, 645(1), 112–141. doi:10.1177/0002716212458082 [Google Scholar]
  34. Smith, W. G. (2008). Does Gender Influence Online Survey Participation? A Record‐linkage Analysis of University Faculty Online Survey Response Behavior. San José State University 2008 Retreived from: http://scholarworks.sjsu.edu/cgi/viewcontent.cgi?article=1003&context=elementary_ed_pub
  35. Spry, V. M. , Hovell, M. F. , Sallis, J. G. , Hofstetter, C. R. , Elder, J. P. , & Molgaard, C. A. (1989). Recruiting survey respondents to mailed surveys: Controlled trials of incentives and prompts. American Journal of Epidemiology, 130, 166–172. doi:10.1177/016327879702000402 [DOI] [PubMed] [Google Scholar]
  36. Steptoe, A. (1989). An abbreviated version of the Miller Behavioral Style Scale. British Journal of Clinical Psychology, 28, 183–184. doi:10.1111/j.2044‐8260.1989.tb00830.x [DOI] [PubMed] [Google Scholar]
  37. Sutherland, K. , Leatherman, S. , & Christianson, J. (2008). Paying the patient: does it work? A review of patient targeted incentives (p. 42). London, UK: QQUIP Quest of Quality and Improved Performance/The Health Foundation; Retrieved from http://www.health.org.uk/sites/health/files/PayingThePatientDoesItWork.pdf [Google Scholar]
  38. Tiro, J. A. , Diamond, P. M. , Perz, C. A. , Fernandez, M. , Rakowski, W. , DiClemente, C. C. , & Vernon, S. W. (2005). Validation of scales measuring attitudes and norms related to mammography screening in women veterans. Health Psychology, 24(6), 555–566. doi:10.1037/0278-6133.24.6.555 [DOI] [PubMed] [Google Scholar]
  39. Tourangeau, R. , & Plewes, T. J. (2013). Nonresponse in Social Science Surveys: A Research Agenda. Washington, D.C.: The National Academies Press. [Google Scholar]
  40. Trussell, N. , & Lavrakas, P. J. (2004). The influence of incremental increases in token cash incentives on mail survey response. Public Opinion Quarterly, 68(3), 349–367. doi:10.1093/poq/nfh022 [Google Scholar]
  41. Ulrich, C. M. , Danis, M. , Koziol, D. , Garrett‐Mayer, E. , Hubbard, R. , & Grady, C. (2005). Does it pay to pay? A randomized trial of prepaid financial incentives and lottery incentives in surveys of nonphysician healthcare professionals. Nursing Research, 54, 178–183. doi:10.1097/00006199‐200505000‐00005 [DOI] [PubMed] [Google Scholar]
  42. Van Kenhove, P. , Wijnen, K. , & De Wulf, K. (2002). The influence of topic involvement on mail‐survey response behavior. Psychology & Marketing, 19(3), 293–301. doi:10.1002/mar.1053 [Google Scholar]
  43. VanGeest, J. B. , Wynia, M. K. , Cummins, D. S. , & Wilson, I. B. (2001). Effects of different monetary incentives on the return rate of a national mail survey of physicians. Medical Care, 39, 197–201. doi:10.1097/00005650‐200102000‐00010 [DOI] [PubMed] [Google Scholar]
  44. Volpp, K. , Levy, A. , Asch, D. , Berlin, J. , Murphy, J. , Gomez, A. , … Lerman, C. (2006). A randomized controlled trial of financial incentives for smoking cessation. Cancer Epidemiology Biomarkers & Prevention, 15(1), 12–18. doi:10.1158/1055‐9965.EPI‐05‐0314 [DOI] [PubMed] [Google Scholar]
  45. Volpp, K. , Troxel, A. , Pauly, M. , Glick, H. , Puig, A. , Asch, D. , … Audrain‐McGovern, J. (2009). A randomized, controlled trial of financial incentives for smoking cessation. New England Journal of Medicine, 360(7), 699–709. doi:10.1158/1055‐9965.EPI‐05‐0314 [DOI] [PubMed] [Google Scholar]
  46. Weinstein, S. J. , Wright, M. E. , Lawson, K. A. , Snyder, K. , Männistö, S. , Taylor, P. R. , … Albanes, D. (2007). Serum and dietary vitamin E in relation to prostate cancer risk. Cancer Epidemiology, Biomarkers & Prevention, 16, 1253–1259. doi:10.1158/1055‐9965.EPI‐06‐1084 [DOI] [PubMed] [Google Scholar]
  47. Young, J. M. , O'Halloran, A. , McAulay, C. , Pirotta, M. , Forsdike, K. , Stacey, I. , & Currow, D. (2015). Unconditional and conditional incentives differentially improved general practitioners’ participation in an online survey: Randomized controlled trial. Journal of Clinical Epidemiology, 68(6), 693–697. doi:10.1016/j.jclinepi.2014.09.013 [DOI] [PubMed] [Google Scholar]

Articles from British Journal of Health Psychology are provided here courtesy of Wiley

RESOURCES