Effect of telephone follow-up on retention and balance in an alcohol intervention trial

Abstract

Objectives

Telephone follow-up is not currently recommended as a strategy to improve retention in randomized trials. The aims of this study were to estimate the effect of telephone follow-up on retention, identify participant characteristics predictive of questionnaire completion during or after telephone follow-up, and estimate the effect of including participants who provided follow-up data during or after telephone follow-up on balance between randomly allocated groups in a trial estimating the effect of electronic alcohol screening and brief intervention on alcohol consumption in hospital outpatients with hazardous or harmful drinking.

Method

Trial participants were followed up 6 months after randomization (June–December 2013) using e-mails containing a hyperlink to a web-based questionnaire when possible and by post otherwise. Telephone follow-up was attempted after two written reminders and participants were invited to complete the questionnaire by telephone when contact was made.

Results

Retention before telephone follow-up was 62.1% (520/837) and 82.8% (693/837) afterward: an increase of 20.7% (173/837). Therefore, 55% (95% CI 49%–60%) of the 317 participants who had not responded after two written reminders responded during or after the follow-up telephone call. Age < 55 years, a higher AUDIT-C score and provision of a mobile/cell phone number were predictive of questionnaire completion during or after telephone follow-up. Balance between randomly allocated groups was present before and after inclusion of participants who completed the questionnaire during or after telephone follow-up.

Conclusion

Telephone follow-up improved retention in this randomized trial without affecting balance between the randomly allocated groups.

Highlights

• •Telephone follow-up was associated with a 20% increase in retention of participants.
• •Younger age and heavier drinking predicted retention by telephone.
• •Providing a mobile/cell number also predicted retention by telephone.
• •Balance between the randomly allocated groups was not affected by telephone follow-up.
• •Telephone follow-up may improve retention in randomized trials.

Background

High attrition rates are common in studies examining the effectiveness of electronic screening and brief intervention for reducing levels of alcohol consumption (Donoghue et al., 2014). This is a concern because attrition, also known as dropout and loss to follow-up, reduces statistical power (Altman, 2007), and can bias the estimates of intervention effects (Bell et al., 2013). Accordingly, our randomized trial testing an electronic alcohol intervention for hospital outpatients with hazardous or harmful drinking was designed to minimize attrition. This included enclosing a monetary incentive with our request to complete the follow-up questionnaire and following up non-completers by telephone. Although the former is recommended in a systematic review of strategies to improve retention rates in randomized trials, the latter is not because only two of the six studies included in the review evaluating the effect of different types of reminders to participants on questionnaire response utilized telephone follow-up (Brueton et al., 2013). Therefore, the aims of this study were to (i) estimate the effect of telephone follow-up on retention, (ii) identify participant characteristics predictive of questionnaire completion during or after telephone follow-up, and (iii) estimate the effect of including participants who provided follow-up data during or after telephone follow-up on balance between randomly allocated groups in an alcohol intervention trial.

Materials and methods

This is a secondary analysis of data from a randomized trial (ACTRN12612000905864) estimating the effect of electronic alcohol screening and brief intervention on alcohol consumption in hospital outpatients with hazardous or harmful drinking (Johnson et al., 2013). Adult outpatients who scored 5–9 inclusive on the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C), a validated 3-item screen for hazardous and harmful drinking (Bradley et al., 2007), were randomized to screening alone or to further assessment and personalized feedback (Johnson et al., 2013). A minimum score of 5 points was selected because a recent Australian study has shown that the optimal cut-off score for the detection of hazardous drinking is ≥ 5 (Vitesnikova et al., 2014). A maximum score of 9 points was selected because the probability of alcohol dependence with an AUDIT-C score above 9 is high (Rubinsky et al., 2010), and these patients probably require more than brief intervention (Saitz, 2010). Alcohol consumption was assessed 6 months after randomization (June–December 2013) by a web questionnaire when the participant provided an e-mail address, and by a paper-questionnaire otherwise. Ethical approval was obtained from the Hunter New England Human Research Ethics Committee (12/05/16/4.04) and the University of Newcastle Human Research Ethics Committee (H-2012-0272).

Strategies that increase survey response rates (Edwards et al., 2009, Millar and Dillman, 2011) and improve retention in prospective cohort studies (Booker et al., 2011) or randomized trials (Brueton et al., 2013) were employed. In brief, participants received a letter on university stationery (with a $20 supermarket voucher) advising them they would receive a follow-up questionnaire in the next few days. Participants who provided an e-mail address were sent a message embedded with a unique hyperlink to the follow-up questionnaire. Participants who did not provide an e-mail address were sent a paper copy. Reminders were sent at 2-week intervals: the first by e-mail or post as per the initial contact and the second by post because offering different response modes sequentially (Web then mail) has been shown to improve response rates (Millar and Dillman, 2011). After a further 2 weeks, telephone follow-up was attempted and participants were invited to complete the questionnaire by telephone.

The following information was collected at baseline: gender, age group, postcode, study group, AUDIT-C score, and contact details for follow-up (postal address, e-mail address, and a telephone number). Postcode was used to determine the Socio-Economic Indexes for Areas (SEIFA) (Australian Bureau of Statistics, 2008). SEIFA percentiles, which divide the distribution of SEIFA scores into 100 equal groups (i.e., the lowest 1% of areas is given a percentile of 1 and the highest 1% of areas is given a percentile number of 100), were used in the analysis.

Statistical analyses

Retention rates were calculated by dividing the number of questionnaires completed by the total number of participants. Multivariable logistic regression was used to identify the participant characteristics predictive of questionnaire completion during or after telephone follow-up among completers. Thus, only participants who completed the questionnaire were included in the logistic regression model and the outcome variable was whether or not the participant completed the questionnaire during or after the telephone follow-up. Chi-square tests and Wilcoxon rank-sum tests were used to assess balance in baseline variables between groups before and after the inclusion of participants who provided follow-up data during or after telephone follow-up.

Results

Participants

The 837 trial participants were aged 18–89 years (37.4% were 18–34 years; 31.7% were 35–54 years; and 30.9% were 55 years or older), 74.9% were men, 76.5% provided a mobile/cell phone contact number, 58.7% provided an e-mail address, and 47.2% were in the intervention group. Median (25th, 75th percentile) AUDIT-C scores and SEIFA percentiles were 6 (5, 8) and 51 (36, 57), respectively.

Retention

The retention rates before and after telephone follow-up were 62.1% (520/837) and 82.8% (693/837), respectively: an increase of 20.7% (173/837). Therefore, 55% (95% CI 49%–60%) of the 317 participants who had not responded after two written reminders responded during or after the follow-up telephone call.

Participant characteristics predictive of questionnaire completion during or after telephone follow-up among completers

The multivariable analysis is shown in Table 1. The odds of questionnaire completion during or after telephone follow-up were higher for participants aged 18–34 and 35–54 years, respectively, compared with participants aged 55 years and older. The odds of questionnaire completion during or after telephone follow-up were also higher for participants who provided a mobile/cell phone number at baseline compared with participants who did not. A one-unit increase in AUDIT-C score at baseline was associated with a 16% increase in the odds of questionnaire completion during or after telephone follow-up.

Table 1.

Odds of completing the follow-up questionnaire during or after telephone follow-up (2013–14) among completers (n = 693).

	Provided follow-up data by web or post (%), N = 520	Provided follow-up data during or after telephone follow-up (%), N = 173	Unadjusted odds ratio (95% confidence interval)	Adjusted odds ratiob (95% confidence interval)
Gender
Women	136 (26.2)	40 (23.1)	1.00	1.00
Men	384 (73.9)	133 (76.9)	1.18 (0.79–1.76)	1.33 (0.86–2.06)
Age group
18–34 years	127 (24.4)	92 (53.2)	5.74 (3.55 − 9.30)	3.98 (2.36 − 6.70)
35–54 years	179 (34.4)	54 (31.2)	2.39 (1.45 − 3.95)	1.75 (1.02 − 3.00)
55 + years	214 (41.2)	27 (15.6)	1.00	1.00
Provided a cell/mobile phone number
No	166 (31.9)	16 (9.3)	1.00	1.00
Yes	354 (68.1)	157 (90.8)	4.60 (2.66 − 7.95)	3.45 (1.93 − 6.19)
Study Group
Control	277 (53.3)	85 (49.1)	1.00	1.00
Intervention	243 (46.7)	88 (50.9)	1.18 (0.84 − 1.67)	1.22 (0.84 − 1.76)
SEIFA percentilea (continuous), median (25th, 75th percentile)	51 (36, 57)	51 (43, 57)	1.00 (0.99 − 1.01)	0.99 (0.98 − 1.00)
Baseline AUDIT-C score (continuous), median (25th, 75th percentile)	6 (5,7)	7 (6, 8)	1.23 (1.09 − 1.40)	1.16 (1.01 − 1.33)

Estimated odds ratios whose 95% confidence intervals do not include 1 are in boldface

^aExcludes 9 participants from areas with no SEIFA score/percentile due to low population.

^bAdjusted for gender, age group, provision of a cell/mobile phone number, study group, SEIFA percentile, and baseline AUDIT-C score.

Balance in baseline variables between the intervention and control groups

Table 2 shows that balance between randomly allocated groups was present before and after the inclusion of participants who completed the questionnaire during or after telephone follow-up.

Table 2.

Balance between randomly allocated groups before and after the inclusion of participants who completed the questionnaire during or after telephone follow-up (2013–14).

	Provided follow-up data by web or post (n = 520)			Provided follow-up data during or after telephone follow-up (n = 173)			All completersd (n = 693)
Intervention (n = 243)	Control (n = 277)	P-value	Intervention (n = 88)	Control (n = 85)	P-value	Intervention (n = 331)	Control (n = 362)	P-value
Male gender, %	72.4	75.1	0.49a	78.4	75.3	0.63a	74.0	75.1	0.74a
Age group,%
18–34 years	24.3	24.6		56.8	49.4		32.9	30.4
35–54 years	33.7	35.0		27.3	35.3		32.0	35.1
55 + years	42.0	40.4	0.93a	15.9	15.3	0.51a	35.1	34.5	0.65a
SEIFA percentile, median (25th, 75th percentile)c	53 (36, 57)	51 (36, 57)	0.89b	51 (43, 57)	51 (37, 57)	0.97b	51 (36, 57)	51 (36, 57)	0.92b
Baseline AUDIT-C Score	6 (5, 7)	6 (5, 7)	0.47b	7 (6, 8)	7 (6, 8)	0.59b	7 (5,8)	6 (5, 8)	0.33b

^aP-value for chi-square test.

^bP-value for Wilcoxon rank-sum test.

^cExcludes 9 participants from areas with no SEIFA score/percentile due to low population.

^dIncludes all participants who provided follow-up data irrespective of mode.

Discussion

Telephone follow-up was associated with a 20% increase in retention. This increase is larger than those reported in a systematic review of the effect of retention methods in population-based cohort studies where reminder calls were associated with increases in retention of 1%–16% (Booker et al., 2011). Plausible explanations for the larger effect seen here include the high proportion of mobile phone ownership (76.5%), which allowed us to contact people even when they were not at home, and our decision to allow participants to complete the follow-up questionnaire by telephone.

Our finding that younger age and heavier drinking at baseline was predictive of questionnaire completion during or after telephone follow-up is consistent with research showing that respondents to an alcohol survey whose participation was the hardest to elicit were younger and reported heavier alcohol consumption (Meiklejohn et al., 2012). Provision of a mobile/cell phone number was also an independent predictor of questionnaire completion during or after telephone follow-up. Given the growth in smartphone ownership (64% of adults in May 2013 compared with 49% in May 2012) and the growth in the number of people accessing the Internet via these devices (7.5 million adults in June 2013 compared with 5.6 million in June 2012) in the 12 months immediately preceding our follow-up (Australian Communications and Media Authority, 2013), it is plausible that some participants who required telephone follow-up had tried to complete the follow-up questionnaire using their smartphone but failed because it was designed for completion on larger-screen devices such as desktop/laptop computers, and tablets. Accordingly, it is also plausible that participants with smartphones would not have required telephone follow-up if the questionnaire had been adapted for smaller screen sizes. There is, however, little research to support this hypothesis. For example, among 979 Dutch adults in possession of a smartphone and with access to the internet who were invited to complete a web-based questionnaire optimized for mobile completion, 55% (536/979) did not respond and only 57% of the respondents (252/443) completed the survey on their smartphone despite encouragement to do so (Toepoel and Lugtig, 2014).

Limitations

The main limitation of the study is the non-randomized design, which precludes a causal inference about the effect of telephone follow-up on retention. It is difficult, however, to think of a plausible alternative explanation for the observed increase. Telephone follow-up also has limitations: this includes the cost (survey research has estimated the cost per completed questionnaire is $27 for telephone reminders and $36 for telephone interviews (Ziegenfuss et al., 2012)) and the possibility that participants who provide follow-up data by telephone may give more socially desirable answers (Couper et al., 2007).

Conclusion

The results of this study support the use of telephone follow-up to improve retention in randomized trials where high attrition is likely.

Conflicts of interest statement

The authors declare that there are no conflicts of interest.

Funding/support

The study was funded by a project grant (APP1023734) from the Australian National Health and Medical Research Council (NHMRC), GPO Box 1421, Canberra, ACT, 2601 (nhmrc@nhmrc.gov.au). KK is supported by a NHMRC Senior Research Fellowship (APP1041867). The NHMRC had no further role in the study design; in the collection, analysis, and interpretation of the data; in the writing of the report; or the decision to submit the paper for publication.