Association Between Interactive Voice Response Adherence and Subject Retention In A Randomized Controlled Trial

Abstract

Introduction

Interactive Voice Response (IVR) technology uses the telephone to collect patient reports. This study examined whether IVR adherence during a year-long clinical trial was related to subject retention in the trial.

Methods

As part of a randomized, double-blind, placebo-controlled study of daily multivitamin supplementation for recurrent aphthous stomatitis, 160 study participants were asked to make 1 weekly IVR call for the one-year study duration.

Results

The 114 subjects who completed the study made 90.5% of their expected number of IVR calls, as compared to 55.7% of expected calls made by the 46 subjects who withdrew prematurely (p<0.001). Subjects who successfully completed the study were also more likely to initiate their IVR calls as compared to subjects who withdrew from the study (p <0.001). A multivariable model incorporating different adherence variables was able to successfully predict retention status of more than 80% of subjects. IVR adherence during the first few weeks of study participation was strongly predictive of subsequent retention and successful completion of this one-year study.

Discussion

Subjects who withdrew prematurely had more missing data than study completers, even after accounting for period of study participation, potentially introducing bias into IVR results. Sub-optimal adherence to weekly IVR might provide an early signal of subsequent premature withdrawal in clinical trials. IVR adherence could be used as a screening tool during a trial period, to identify subjects most likely to stay on long clinical trials.

Conclusion

IVR adherence may be useful in anticipating retention in long-term clinical studies.

Introduction

Interactive Voice Response (IVR) is an automated data collection tool in which study participants use their telephone keypad to answer pre-recorded questions. IVR is especially useful in long term studies involving large numbers of subjects. Data collection is less time and labor intensive for investigators, and less demanding for study participants, who can respond to the questionnaires remotely from a convenient location. Due to its many potential advantages, IVR is being increasingly used in clinical research [1, 2, 3].

The relative costs and benefits of electronic data capture such as IVR compared to paper reports completed off site has been debated. In a previous study of head and neck cancer patients undergoing radiation therapy [4], we demonstrated high concordance between paper diaries and IVR. However, paper diaries yielded greater adherence, though adherence indicators for paper diaries are less verifiable. Whereas some studies comparing paper and electronic diaries have concluded that electronic diaries are superior; others have found no difference or have presented a mixed picture of the relative merits of each method [5, 6]. It has been suggested that either method can be suitable for studies where discrete behaviors are being studied. On the other hand, electronic diaries may be preferable for examining within-day temporal dynamics or evaluating rapidly changing phenomena [7].

In the current study we used weekly IVR patient reports in a year-long clinical trial to evaluate whether IVR adherence was an anticipatory marker of a participant’s premature withdrawal from the study. We also examined whether IVR adherence in the first few weeks of study participation may predict subsequent retention on the study.

Methods

Subjects

This is a secondary analysis of data collected during a randomized, double-blind, placebo-controlled clinical trial of multivitamin supplementation for Recurrent Aphthous Stomatitis (an idiopathic ulcerative condition of the oral mucosa) [8]. The trial was conducted at the University of Connecticut Health Center, Farmington, CT, USA. 160 subjects were randomized to either a multivitamin supplement or placebo in a 1:1 ratio, and asked to take the study medication once a day for the 1 year duration of the study.

The sample size of 160 subjects was calculated based on the primary objectives of the parent clinical trial i.e. to evaluate the effect of daily multivitamin therapy on the number and duration of Recurrent Aphthous Stomatitis episodes. Specifically, we calculated study sample size based on 80% power, and 0.05 alpha, to detect at least a 65% probability that a randomly chosen person from the multivitamin arm has a superior outcome (fewer and/or shorter episodes) when compared to a randomly chosen person from the placebo arm. Based on these parameters, we estimated that a final sample size of at least 100 subjects (50 per arm) would be needed to achieve adequate power, after accounting for dropouts. Since this was a one-year study, we allowed for a relatively high drop-out rate by enrolling 160 subjects. One hundred and fourteen participants completed the year-long study and 46 withdrew before completing the study. The study was approved by the Institutional Review Board and all subjects provided written informed consent.

IVR Data Collection

Subjects were asked to call the IVR system (IVRS) once a week (Monday to Sunday) for the duration of the study, with a week’s period between successive calls. However, the IVRS was accessible to subjects at all times during study participation. If the IVRS had not received the call by 7 pm on Thursday of a given week, the subject received an automated reminder call, which also allowed him/her to complete the survey at that time. A second automated reminder call was made at 8 pm on Friday if the subject had still not called the IVRS. The IVR survey asked subjects if they had experienced any mouthsores, how many doses of study medication they had missed and about any illness or unusual stress experienced, all since the last call. Subjects responded to these questions by pressing numbers on the telephone keypad. On average, each call took about 1–2 minutes. Subjects were compensated US$5 for each weekly IVR call completed over the course of the study.

IVR Adherence and Subject Retention

The IVRS tracked the number of weekly IVR surveys completed by each subject, and whether the subject called the system or completed the survey during a system-initiated reminder call. For subjects who successfully completed the year-long study, the total number of expected calls was 52. For subjects who dropped out of the study, the total number of expected calls was the number of full weeks elapsed from the baseline visit to the date the subject made his/her last IVR call. The actual number of calls made by each subject was compared to the number of expected calls for that subject to calculate a compliance rate for each subject. A mean compliance rate was calculated for all 114 subjects who successfully completed the study and separately for all 46 subjects who dropped out. These compliance rates were compared using the generalized linear model for a binary outcome. This analytic approach was needed to be able to properly weight each subject by the number of observations provided (i.e. duration of study participation). We also examined the relationship between subject retention and the proportion of IVR surveys completed by each subject that were initiated by the subject, versus completing the survey during a system-initiated reminder call, using a chi square test. Further, the relationship between subject retention and promptness in completing the IVR calls was examined using a chi square test i.e. whether the survey was completed before the first reminder call or not. A multivariable model was used to examine the combined relationship of IVR adherence, call initiation, and call promptness, on subject retention. To examine the predictive utility of IVR adherence on subject retention during the first few weeks of the study, the proportion of IVR surveys done by completers and non-completers during their first six weeks of study participation was compared, using a chi square test.

Results

Interval between Successive IVR Calls

Subjects were generally compliant with the instruction to maintain a week’s period between IVR calls. For all IVR calls completed in successive weeks, the mean period between calls was 7.82 days. 76.5% of all calls occurred between 6 and 8 days of the previous call by the same subject. Only 2.9% of calls occurred within 4 days of the previous call by the same subject.

IVR Adherence and Subject Retention

The 114 subjects who completed the study made 90.5% of expected IVR calls as compared to 55.7% for the 46 subjects who withdrew from the study. Subjects who completed the study were more likely to complete their IVR calls compared to subjects who withdrew (p < 0.001), even after accounting for study condition (multivitamin vs placebo) and severity of illness (as measured by number of episodes of recurrent aphthous stomatitis reported by study subjects, over 70% of these episodes were also verified by clinical examination). Among subjects who withdrew from the study prematurely, exact duration of study participation showed no relationship to IVR adherence (p=0.48). Subjects who completed the study were also more likely to initiate their IVR calls (79.4% of completed calls) compared to subjects who withdrew (71.6%, p <0.001). Among the subjects who completed the study, 67.4% of calls were made promptly (defined as before the first reminder call), while among the subjects who withdrew prematurely, 60.1% of the calls were made promptly (p = 0.106). A multivariable model incorporating all 3 compliance variables (i.e. call adherence, call initiation, and call promptness) was able to correctly predict retention status of 80.5% of subjects. Of the 3 compliance variables in the multivariable model, the strongest individual predictor of retention status was call adherence (p<0.001). Call initiation alone (p=0.929) and call promptness alone (p=0.339) did not significantly predict retention status in this model.

Table 1 shows the cumulative adherence with completing IVR calls in the first 6 weeks of the study for study completers and non-completers. This period represents approximately the first 10% of the 52-week study duration. At each weekly time-point, there was a statistically significant difference between the two groups in adherence to completing IVR calls. Although cumulative adherence dropped in both groups over the 6 week period, the drop was much more pronounced in the subjects who ultimately withdrew from the study. 10 subjects made 1 call or less during the first 4 weeks of study participation; of these, 9 subjects did not complete the study, with 7 of them withdrawing after the first 4 weeks.

Table 1.

Cumulative IVR Call Adherence by Week for Study Completers and Non-Completers

Week	Number of subjects who completed all expected IVR calls (% of total number of subjects in that group)		P-Value
	Completers (n=114)	Non-completers (n=46)
1	114 (100%)	40 (87%)	<0.001
2	109 (96%)	33 (72%)	<0.001
3	103 (90%)	29 (63%)	<0.001
4	101 (89%)	28 (61%)	<0.001
5	96 (84%)	21 (46%)	<0.001
6	89 (78%)	21 (46%)	<0.001

Note: IVR adherence presented at each week is cumulative i.e. data for each weekly time-point includes all IVR calls up to that time-point.

Discussion

IVR is being increasingly used for data collection due to its many advantages including timestamped data collection (as compared to paper forms which can be backfilled). We found that subjects who withdrew prematurely from the study completed fewer IVR calls (and therefore had more missing data) than those who completed the study, even after accounting for the period of study participation. This can introduce a potential source of bias into IVR results. However, a similar phenomenon may occur with other forms of data collection such as paper diaries; this is a fertile area for future study.

Subject retention in lengthy clinical trials is a significant challenge faced by researchers in all clinical fields. We found that weekly IVR adherence predicted completer status. A multivariable model incorporating different adherence variables was able to successfully predict retention status of more than 80% of subjects. Interestingly, IVR adherence during the first few weeks of study participation was strongly predictive of subsequent retention and successful completion of this one-year study. These findings indicate that IVR adherence could be used as a screening tool during a trial period of a few weeks, to identify subjects most likely to stay on long clinical trials. This would in turn save significant resources and speed up study completion. However, it should be noted that such selection of subjects may introduce some selection bias which could affect the external validity of the study results. Additional studies, including studies with non-IVR control groups, may help determine whether IVR and other electronic data capture methods not only predict, but also promote subject retention. Future studies should also measure person factors such as conscientiousness and motivation as possible third factors in the associations we report here. Finally, additional research is needed to determine whether these results generalize to other populations.

Conclusion

Our findings suggest that adherence to IVR and other electronic data capture methods may predict the subsequent completer status of individuals in lengthy clinical trials.