Differences in response rates between mail, e-mail, and telephone follow-up in hand surgery research

Abstract

Background

There is a need to determine the difference in response to mail, e-mail, and phone in clinical research surveys.

Methods

We enrolled 150 new and follow-up patients presenting to our hand and upper extremity department. Patients were assigned to complete a survey by mail, e-mail, or phone 3 months after enrollment, altering the follow-up method every 5 patients, until we had 3 groups of 50 patients. At initial enrollment and at 3 month follow-up (range 2–5 months), patients completed the short version of the Disabilities of the Arm, Shoulder, and Hand questionnaire (QuickDASH), the short version of the Patient Health Questionnaire (PHQ-2), the Pain Self-Efficacy Questionnaire (PSEQ), and rated their pain intensity.

Results

The percent of patients that completed the survey was 34 % for mail, 24 % for e-mail, and 80 % for phone. Factors associated with responding to the survey were older age, nonsmoking, and lower pain intensity. Working full-time was associated with not responding.

Conclusions

The response rate to survey by phone is significantly higher than by mail or e-mail. Younger age, smoking, higher pain intensity, and working full-time are associated with not responding.

Type of study/level of evidence: Prognostic I

Introduction

Follow-up surveys are often used for long-term evaluation of patients in clinical research. The response to mail surveys is often very low raising the concern of bias if responders are different from nonresponders. In our survey studies, about a third of patients return the mailing [4, 8, 9]. Previous studies suggested nonresponders are different from responders with regard to age, gender, pain intensity, marital status, working status, smoking habit [1, 5], and satisfaction with treatment [6]. We wondered if phone or e-mail might improve the response rate.

This study addressed the primary null hypothesis that there is no difference in response rate between three types of follow-up (mail, e-mail, and phone). Secondary analyses assessed the differences between responders and nonresponders for each follow-up type.

Material and Methods

One hundred and eighty-six new and follow-up patients were invited to participate in this institutional review board-approved study during their regular outpatient visit to the hand surgeon. We excluded patients that were pregnant, had no access to internet, or used internet less than once a week. All patients provided informed consent for this prospective cohort study and were enrolled between November 2012 and January 2013. One patient withdrew from the study during the completion of the questionnaires, 1 patient was excluded due to technical malfunction of the electronic data capturing device during initial enrollment, and 34 patients declined to participate. This resulted in a study sample of 150 patients.

To determine the response rate of different follow-up methods, we asked patients to complete the follow-up questionnaires at 3 months (acceptable range between 2 and 5 months) after the initial enrollment. Subjects were assigned to three different follow-up methods: mail, e-mail, or phone, altering the method of follow-up every five enrolled patients. This resulted in three groups containing 50 patients.

Patients that were assigned to follow-up by mail were sent a postal survey once, containing a letter with instructions, the questionnaires, and a stamped return envelope. The patients that were assigned to follow-up by e-mail were sent one e-mail with instructions and an internet link to the Research Electronic Data Capture (REDCap) survey containing the questionnaires. Patients assigned to follow-up by phone were called on their landline or cell phone, with a maximum of three attempts. There was no difference in the type of questionnaires presented to the patients in the different follow-up groups.

Measurements

At initial enrollment, demographics were recorded and subjects completed the questionnaires. The short version of the Disabilities of the Arm, Shoulder, and Hand questionnaire (QuickDASH) was used to measure disability (scores between 0–100, where higher scores represent more disability). Additionally, patients were asked to complete the short version of the Patient Health Questionnaire (PHQ-2) to measure depressive symptoms and the Pain Self-Efficacy Questionnaire (PSEQ) to measures the patient’s sense that they can accomplish their goals in spite of pain. Furthermore, subjects rated their pain on an 11-point ordinal scale from 0 (no pain) to 10 (worst pain ever). At follow-up, patients were asked to complete the QuickDASH, PHQ-2, PSEQ, and 11-point ordinal measures of pain intensity and satisfaction with the visit to the hand surgeon.

Statistical Analysis

An a priori sample size calculation with a Chi-square test determined that 143 patients were needed to provide 80 % power with alpha 0.05 to detect a medium effect size (d = 0.3) difference in response rate between the three follow-up methods. We accounted for 5 % incomplete data and enrolled 150 (50 patients per group) patients.

Statistical significance of the difference in follow-up rate between the three follow-up methods (mail, e-mail, or phone) was assessed using the chi-square test. Statistical significance of the differences in other discrete variables was determined using chi-square test or Fisher’s exact test, if applicable. Differences in continuous variables between the three follow-up groups were assessed with one-way ANOVA or Kruskal-Wallis if the variable was nonparametric. Differences in continuous variables between responders and nonresponders were determined with Student’s t test or the Mann–Whitney U (nonparametric) test. Variables with a P value less than 0.10 in bivariate analysis were entered in multivariable logistic regression analysis.

Results

Patients

The patients had a mean age of 49 ± 17 years (range 19 to 92 years). The slight majority of the sample consisted of women (52 %). One hundred patients were established patients in our practice being seen in regular follow-up (67 %), and 50 patients were new patients to our practice (33 %). Of the 150 patients enrolled in the study, 69 (46 %) responded to our request for follow-up by any method at 3 months (Table 1).

Table 1.

Patient Demographics

	Cohort n = 150			Responders n = 69			Nonresponders n = 81
Parameter	Mean	SD	Range	Mean	SD	Range	Mean	SD	Range	P value
Age (years)	49	17	19–92	53	17	19–84	46	17	19–92	0.0046
Education (years)	16	2.5	10–22	16	2.5	10–22	16	2.4	12–20	0.37
Parameter	Number		%	Number		%	Number		%
Gender
Male	72		48	36		52	36		44	0.35
Female	78		52	33		48	45		56
Type of visit
New patient	50		33	25		36	25		31	0.49
Follow-up	100		67	44		64	56		69
Sought treatment for condition before
Yes	48		32	18		26	30		37	0.15
No	102		68	51		74	51		63
Other pain conditions
Yes	48		32	24		35	24		30	0.50
No	102		68	45		65	57		70
Prior surgery
Yes	19		13	10		14	9		11	0.54
No	131		87	59		86	72		89
Marital status
Single	57		38	22		32	35		43	0.56
Living with partner	2		1.3	1		1.5	1		1.2
Married	68		45	36		52	32		40
Separated/Divorced	15		10	6		8.7	9		11
Widowed	8		5.3	4		5.8	4		4.9
Working status
Full-time	93		62	35		51	58		72	0.041
Part-time	10		6.7	6		59	4		4.9
Homemaker	1		0.67	0		0	1		1.2
Retired	24		16	17		25	7		8.6
Unemployed, able to work	4		2.7	3		4.4	1		1.2
Unemployed, not able to work	10		6.7	5		7.3	5		6.2
On worker’s compensation	2		1.3	0		0	2		2.5
Currently on sick leave	6		4.0	3		4.4	3		3.7
Smoking
Yes	17		11	3		4.4	14		17	0.018
No	133		89	66		96	67		83

Bivariate Analysis

Each follow-up group (mail, e-mail, and phone) included 50 patients. The response rate was 34 % (17 patients) in the mail group, 24 % (12 patients) in the e-mail group, and 80 % (40 patients) in the phone group, these differences were significant (P < 0.001) (Tables 2 and 3). Patients responding to follow-up (n = 69) were significantly older than nonresponders (53 vs. 46, P = 0.0046) (Table 1). Significantly, more patients smoked in the nonresponder group (P = 0.018). There was a significant difference in work status between responders and nonresponders: there were significantly less patients working full-time in the group of responders (P = 0.041). There was no significant difference between the other demographics (education, gender, type of visit, sought treatment before, prior surgery, marital status) between responders and nonresponders (Table 1).

Table 2.

Response rate of the 50 patients for each response group

	Cohort n = 150		Responders n = 69		Nonresponders n = 81
Parameter	Number	%	Number	%	Number	%	P value
Type of follow-up
Phone	50	33	40	58	10	12	<0.001
Mail	50	33	17	25	33	41
E-mail	50	33	12	17	38	47

Table 3.

Difference in the response rate of each response group

	Cohort n = 150		Phone n = 50		Mail n = 50		E-mail n = 50
Parameter	Number	%	Number	%	Number	%	Number	%
Responder
Yes	69	46	40	80	17	34	12	24
No	81	54	10	20	33	66	38	76

Patients not responding to follow-up were experiencing significantly more pain at the time of enrollment (P = 0.0079) (Tables 4, 5, and 6). Disability, depression, and pain self-efficacy were not significantly different between responders and nonresponders (Table 6). In the group of patients who were approached by e-mail and responded to follow-up, none of the patients reported other pain conditions (P = 0.0040) (Table 4). There were no other significant differences between the three follow-up groups (Tables 4 and 5).

Table 4.

Patients not responding to follow-up who were experiencing significantly more pain at the time of enrollment

Responders n = 69	Phone n = 40		Mail n = 17		E-mail n = 12
Parameter	Mean	SD	Mean	SD	Mean	SD	P value
Age	51	18	53	16	60	14	0.15
Education	16	2.7	16	2.5	16	1.9	0.44
Pain	3.8	2.9	3.1	2.8	2.5	2.9	0.34
Pain (at follow-up)	2.4	2.7	1.5	2.0	1.6	2.3	0.59
QuickDASH	31	21	29	23	21	17	0.29
QuickDASH (at follow-up)	19	23	15	18	18	20	0.70
PHQ-2	0.60	1.1	1.0	1.7	0.17	0.58	0.41
PSEQ	49	13	51	9.1	54	6.4	0.26
Satisfaction with condition (at follow-up)	7.7	2.4	8.1	2.4	7.6	2.4	0.65
Satisfaction with treatment (at follow-up)	7.9	2.9	9.2	1.2	8.2	3.3	0.53
Overall health	8.2	1.5	7.8	2.3	8.1	1.9	0.94
	Number	%	Number	%	Number	%
Gender
Male	20	50	11	65	5	42	0.43
Female	20	50	6	35	7	58
Type of visit
New	17	43	4	24	4	33	0.40
Follow-up	23	58	13	76	8	67
Sought treatment for condition before
Yes	11	28	5	29	2	17	0.81
No	29	73	12	71	10	83
Other pain condition
Yes	19	48	5	29	0	0	0.0040
No	21	53	12	71	12	100
Prior surgery
Yes	6	15	2	12	2	17	0.99
No	34	85	15	88	10	83
Marital status
Single	13	33	7	41	2	17	0.73
Living with partner	1	2.5	0	0	0	0
Married	19	48	9	53	8	67
Separated/Divorced	5	13	0	0	1	8.3
Widowed	2	5.0	1	5.9	1	8.3
Working status
Full-time	21	53	10	59	4	33	0.50
Part-time	4	10	1	5.9	1	8.3
Homemaker	0	0	0	0	0	0
Retired	7	18	3	18	7	58
Unemployed, able to work	2	5	1	5.9	0	0
Unemployed, not able to work	4	10	1	5.9	0	0
On worker's compensation	0	0	0	0	0	0
Currently on sick leave	2	5	1	5.9	0	0
Smoking
Yes	3	7.5	0	0	0	0	0.75
No	37	93	17	100	12	100

Table 5.

Differences between the three follow-up groups

Nonresponders n = 81		Phone n = 10		Mail n = 33		E-mail n = 38
Parameter		Mean	SD	Mean	SD	Mean	SD	P value
Age		42	16	50	18	43	15	0.28
Education		16	2.8	15	2.5	16	2.3	0.66
Pain		4.3	2.8	4.2	2.4	4.9	2.8	0.47
QuickDASH		26	23	32	18	37	24	0.38
PHQ-2		0.30	0.48	0.33	0.65	0.68	1.5	0.98
PSEQ		53	7.5	49	10	49	12	0.38
		Number	%	Number	%	Number	%
Gender	Male	3	30	16	48	17	45	0.64
	Female	7	70	17	52	21	55
Type of visit
New patient		5	50	11	33	9	24	0.26
Follow-up		5	50	22	67	29	76
Sought treatment
Yes		5	50	9	27	16	42	0.29
No		5	50	24	73	22	58
Other pain condition
Yes		1	10	9	27	24	63	0.25
No		9	90	24	73	14	37
Prior surgery
Yes		1	10	5	15	3	7.9	0.70
No		9	90	28	85	35	92
Marital status
Single		6	60	11	33	18	47	0.76
Living with partner		0	0	0	0	1	2.6
Married		3	30	15	45	14	37
Separated/divorced		1	10	4	12	4	11
Widowed		0	0	3	9.1	1	2.6
Work status
Full-time		10	100	21	64	27	71	0.94
Part-time		0	0	1	3.0	3	7.9
Homemaker		0	0	1	3.0	0	0
Retired		0	0	4	12	3	7.9
Unemployed, able to work		0	0	1	3.0	0	0
Unemployed, not able to work		0	0	2	6.1	3	7.9
On worker's compensation		0	0	1	3.0	1	2.6
Currently on sick leave		0	0	2	6.1	1	2.6
Smoking
Yes		4	40	6	18	4	11	0.090
No		6	60	27	82	34	89

Table 6.

Disability, depression, and pain between responders and nonresponders

	Responders n = 69		Nonresponders n = 81
Parameter	Mean	SD	Mean	SD	P value
QuickDASH	29	22	34	21	0.16
PHQ-2	0.62	1.2	0.49	1.1	0.56
PSEQ	50	11	49	11	0.53
Pain	3.4	2.7	4.6	2.7	0.0079

Multivariable Analysis

Working status, age, smoking, pain intensity, and the type of follow-up were entered in a logistic regression model looking at predicting factors for responding to follow-up. Working full-time (OR 0.26; CI 0.11–0.64; P = 0.0030) was associated with no response. Nonsmoking (OR 9.1; CI 1.9–45; P = 0.0060), lower pain intensity (OR 1.2; CI 1.0–1.4; P = 0.021), and follow-up by phone (OR 18; CI 6.3–51; P = <0.001) were associated with responding to follow-up.

Discussion

We were able to reject the primary null hypothesis by demonstrating a difference in response rate between follow-up by mail, e-mail, or phone. We found that 80 % of patients approached by phone responded (OR 18) to follow-up. Response rates of follow-up by mail or e-mail were 34 and 24 %, respectively. Factors associated with responding to follow-up were nonsmoking (OR 9.1) and lower pain intensity (OR 1.2). Working full-time was associated with nonresponding to follow-up (OR 0.26).

These results should be viewed in light of the limitations of this study. First, patients assigned to follow-up by either mail or e-mail were sent a letter or e-mail once, whereas patients assigned to follow-up by phone were approached until the patient responded, with a maximum of three attempts. Since we excluded patients with limited or no internet access, the actual response rate might even be lower in a random patient selection. In the phone group, we decided to use multiple attempts, thereby taking into account that a phone call can be missed (in contrast to a letter or e-mail). The number of attempts could have influenced the large response rate observed in the phone follow-up group; however, 42 % of patients responded after only a single attempt. The response rate after only a single attempt still exceeds that of either the e-mail or the mail group. We did not send out (multiple) reminder e-mails to adjust for the possibility of patients losing track of e-mails in the modern high-frequency e-mail traffic. It is possible repeat e-mails would have increased the response rate in this group. The response rate for follow-up by mail is consistent with the percentages in other studies [1, 8, 9]. The higher response rate for follow-up by phone is in line with earlier research [11]. Different response rates in literature could be present due to different study design [3, 7]. Rhee et al. [11] assigned emergency department patients to either follow-up by phone or mail and reported response rates of 41 and 20 %, respectively. The results of our study might vary with the number of questionnaires that patients are asked to complete. Second, this study is performed in subjects with various diagnoses presenting to an orthopedic hand service. A study in a different population (e.g., just one discrete diagnosis or in a different medical field) might result in different findings. Despite this possible variation, previous research found comparable response rates between patients with different diagnoses and in other medical specialties [11].

Earlier studies reported that younger age, male sex, more pain, working status, marital status, and smoking were associated with not responding to follow-up [1, 2, 5, 10, 12]. Apart from gender and marital status, we found the same predictors of nonresponse to follow-up. Of note however; in multivariable analysis, we found that age is not an independent predictor of responding. Working full-time is associated with nonresponding, which could be explained by the shortage of time and other priorities.

The increased use of the internet in recent decades might imply that follow-up by e-mail could result in higher reponse rates. On the contrary, this study found higher response rates in patients approached for follow-up by phone. Future studies using questionnaires for the long-term evaluation of patients could consider using follow-up by phone, to maximize response rate. The results of this investigation suggest that future studies designed using questionnaires for short-term evaluation can maximize their response rate using a phone follow-up method. It is not known if this holds true for periods of follow-up longer than 3 months. However, follow-up by phone is potentially more time consuming and costly.

Furthermore, if studies are performed in populations associated with nonresponding (e.g., younger age, male gender, smoking, working full-time), low response rates should be taken into account. Since follow-up by e-mail is an easy, cheap, and environmentally friendly follow-up method, future studies about the response rates of e-mail could consider using reminders to improve response rates. Lastly, it could be beneficial to study if the quantity, extent, and content of the questionnaires (e.g., questionnaires with less personal information) influences the follow-up rate.