Dermatologist response rates to a mailed questionnaire: a randomized trial of monetary incentives

To the Editor:

Although surveys are frequently used to collect data from dermatologists, response rates are often low, limiting the generalizability of results.¹ Monetary incentives have improved physician survey response but have not been tested in dermatologists.² Moreover, the effect of incentive size remains unclear.² This study examines the effect of cash incentives on dermatologist response to a mailed questionnaire and its cost-effectiveness.

As part of a study about preferences for psoriasis treatment, we surveyed 1000 dermatologists,³ randomizing each to receive an initial questionnaire packet with either $5 or $10 (with a note offering this token of appreciation) or no cash (Figure 1). Using a modified Dillman Tailored Design method,⁴ we sent postcard reminders and duplicate surveys to non-respondents after the initial mailing. The study was approved by the University of Pennsylvania Institutional Review Board.

FIGURE 1. Flow chart of study design and outcome.

One thousand dermatologists were randomly selected from American Academy of Dermatology members who self-identified as treating psoriasis and active National Psoriasis Foundation dermatologist members. Dermatologists within both groups were stratified and equally randomized to receive one of three incentive amounts. Six subjects were unreachable due to undeliverable mailings, and 5 subjects were considered ineligible because they were non-dermatologists or not currently in practice. Thirteen eligible subjects who returned the survey unanswered were treated as non-respondents. AAD, American Academy of Dermatology; NPF, National Psoriasis Foundation.

We compared survey response with respect to physician characteristics and incentives and performed logistic regression to evaluate interactions among possible predictors of response determined a priori. We calculated cost per response and incremental cost-effectiveness, considering only incentive and material costs ($1/questionnaire without incentive, $1.10/questionnaire with incentive, $0.75/postcard, $0.50/response postage), and performed sensitivity analysis by maximizing material costs. Confidence intervals (CI) were calculated using bootstrap and Fieller theorem methods.

The overall response rate was 39.1%, with rates of 25%, 43% (odds ratio (OR) 2.26, 95% CI 1.61-3.16), and 49% (OR 2.80, 95% CI 1.99-3.93) in the $0, $5, and $10 groups, respectively (Figure 1). However, response rates in the $5 and $10 groups did not differ significantly (p=0.17, Fisher’s exact). In multivariate logistic regression, NPF membership (OR 2.48, 95% CI 1.89-3.26) and receipt of incentive (OR 2.63, 95% CI 1.94-3.55) were significant predictors of survey response, while sex and duration of practice were non-significant.³

Inclusion of $5 instead of $0 or of $10 rather than $5 cost $27.35 (95% CI 19.33-45.93) and $88.83 (37.00-(−244.89)) per extra response, respectively; the latter CI indicates that at an investigator’s willingness-to-pay between $0 and $37 per extra response, we can be 95% confident that $10 incentives represent bad value compared to $5 incentives. In the sensitivity analysis, once questionnaire and postcard costs surpassed $1.88 and $1.63, respectively, the cost per response in the $0 group exceeded that in the $5 group; at these material costs and higher, the use of $5 incentives, compared to no incentive, thus represents better value when investigators are concerned with increasing the total number of responses.

Although dermatologists have generally low response rates to surveys, they respond just as positively to monetary incentives as other specialists.² The higher response among NPF members also supports previous findings that topical salience influences survey response.^{2, 5} Moreover, financial incentives can be cost-effective, particularly when survey costs are high. However, increasing incentive size leads to diminishing returns and amounts over $5 may be less cost-effective.⁵

To engage dermatologists in survey-based research, investigators should consider using small financial incentives. Future studies should investigate response rates in other surveys of dermatologists to further assess the generalizability of our results.

TABLE I.

Cost per completed survey and incremental cost-effectiveness by incentive group

Cost	Incentive
	$0	$5	$10
Total cost	$1082.50	$2719.65	$4348.55
Cost/response1 (95% CI)	$13.04 (10.87, 16.20)	$19.02 (16.85, 21.79)	$27.01 (24.15, 30.60)
Incremental cost-effectiveness2 (95% CI)	-	$27.35 (19.33, 45.93)	$88.83 (37.00, −244.89)
Cost/response at maximal costs3 (95% CI)	$41.77 (34.64, 52.14)	$34.15 (29.99, 39.46)	$39.87 (35.36, 45.53)
Incremental cost-effectiveness at maximal costs (95% CI)	-	$23.54 (15.72, 41.59)	$84.08 (33.10, −243.54)

¹Cost per response was calculated by dividing the total expenditures per incentive group by the number of returned surveys in that group.

²Incremental cost-effectiveness was calculated by dividing the mean total cost per subject between two incentive groups by the difference in response rate between the two groups.

³In our sensitivity analysis, maximal costs were calculated by adding $2 to both the questionnaire and postcard mailing costs per person.

ABBREVIATIONS

AAD

American Academy of Dermatology

CI

confidence interval

IQR

interquartile range

NPF

National Psoriasis Foundation

OR

odds ratio

SD

standard deviation