Tracking and Locating Itinerant Subjects with a Rechargeable Incentive Card: Results of a Randomized Trial

Low follow-up rates in field research raise serious questions regarding the validity and generalizability of results (Crisanti et al., 2014; Kleber & D’Aunno, 2001). In general, the harder it is to find subjects at follow-up, the more impaired they are (with regard to drug use and criminality). Nemes’ study of cocaine users had a 23% follow-up rate from the first contact attempt, increasing to 54% when up to four attempts were made, and up to 80% with nine attempts (Nemes et al., 2002). Comparing outcome results by follow-up percentage showed cocaine use at follow-up was underestimated and employment at follow-up was overestimated for findings derived from less than 80% of the target sample. Interviewing a self-selected subsample at follow-up is likely to bias findings in favor of clients who are located more easily or whose lives are more stable. Hard-to-track individuals, therefore, may be more deviant, sick, or have poorer outcomes than those respondents who can be contacted more readily.

Estimating the costs specifically associated with tracking and locating study subjects is complex. In our literature review we were unable to locate any empirical studies that directly addressed this topic. The most useful estimate was reported by Hansten at al. (2000) in their study of the relationship between follow-up rates and substance abuse treatment outcomes. These investigators found that the marginal costs associated with increasing their follow-up rates from 60% to 90% (where N=654) was approximately $40,000.

Evidence for using debit-card technology to enhance follow-up rates

Over the past decade, the number of payments issued by check in the United States has declined at an annual rate of 3% to 4%. In fact, by 2003, the number of electronic payments exceeded the number of check payments for the first time—fueled primarily by the rapid growth in debit card payments (Federal Reserve System, 2004). The reason for the widespread adoption of this technology is clear: relative to the use of checks—or even cash in some contexts—debit cards offer a faster, simpler way of making financial transactions. And, relative to credit cards, debit cards protect against spending beyond one’s means. In spite of the popularity and obvious benefits of debit cards, their adoption by longitudinal researchers has been slow. Our review of the literature revealed only one study that examined the use of debit cards as a means of enhancing follow-up rates in a longitudinal examination of homeless drug users. In this study, De Jarlais, Perlis, and Settembrino (2005) issued debit cards to a sample of 139 “urban nomads” and made deposits to their accounts each time they completed a telephone interview. The authors found that subjects with debit cards had substantially higher follow-up rates than similar subjects in two related studies that relied on traditional methods of issuing subject payments (81% vs. 31% and 67% at 6 months; 71% vs. 10% and 41% at 12 months).

Further evidence supporting the use of payment cards for substance abusers participating in longitudinal research comes from prior behavioral research with this population—particularly with regard to the effects of contingency management (CM). One of the critical elements identified in successful implementation of a CM protocol is the immediacy of the reinforcer. Rowan-Szal and colleagues (1994) found that clients who had to wait for a long period of time before they could exchange points for retail items achieved lower levels of abstinence than those who were able to exchange their points immediately. Indeed, laboratory research has shown that substance abusers have a particularly truncated sense of time and tend to discount their perceived value of delayed rewards at a rate of 2 to 4 times that of non-substance abusers (Petry & Casarella, 1999). Consequently, the need for immediate reinforcement is especially acute for this population, offering further support for the potential value of instantly creditable payment methods in longitudinal research with substance abusers.

The purpose of this experimental study was to assess whether a rechargeable incentive card (RIC) is more effective than money orders in keeping study participants engaged in follow-up activities and to determine if there are any cost savings associated with using the RIC payment system.

METHODS

Sample

Participants were recruited from three Los Angeles-based substance abuse treatment programs—two outpatient and one residential. All clients of the participating substance abuse treatment programs were eligible to enroll in this study. Among those invited to participate, 77% agreed to do so. Slightly over half of the participants were male (56%) and the average age was 37. The majority described themselves as white-non-Hispanic (54%), followed by Hispanic (24%), and African American (12%). One third categorized themselves as homeless. Current legal status was not directly assessed, but 16% of the sample were drug court participants under probation/parole supervision.

Procedures

All procedures for this study were reviewed and approved by the UCLA Institutional Review Board. Enrollment for this study took place from January 2013 to September 2013. During the enrollment period, research staff introduced the study to patients attending group treatment sessions at the participating treatment programs. Treatment clients who were interested in learning more about the study were invited to meet with a member of the research team immediately following the group treatment session. The treatment clients who met with a team member to learn more about the study were informed that they were being asked to participate in a study that is assessing the effectiveness of the RIC payment system in improving follow-up rates when compared to a money order payment system. Participants who consented to be a part of this study were asked to complete a baseline and follow-up interview. The baseline interview was conducted as soon as the participant consented to be in the study and elicited data on demographics and deferment of gratification. The follow-up interview, which was conducted 6 months later over the phone, assessed the participants’ experience and satisfaction with the payment system they were assigned to. Participants were paid $30 for completing the baseline interview and $30 for completing the follow-up interview. All participants were asked to call the research team each month to update their contact information during the 5-month period between the baseline and follow-up interview, with each call resulting in a $10 payment.

Participants were given a card that contained the study phone number and the schedule of activities they were expected to complete (i.e. monthly calls and final interview) that included the due dates. Thus, participants were expected to automatically call the research team to complete their monthly call activity or final interview. If a study participant did not call in during the scheduled time for a study activity, research assistants would follow up with the study participant to remind them to complete the next monthly call or to conduct the final interview. Since a portion of the study sample was homeless, the study team took measures to eliminate barriers that would make it difficult for them to make phone calls or receive their payment in the mail. Specifically, the research team set up a toll-free number for the monthly calls so that all subjects could call the research center from a pay phone or any other mobile device to update their locator information. Moreover, RIC participants were provided with a toll-free number on the back of the RIC card to find out their card balance and other relevant information regarding their payments. Additionally, all participants were asked to list at least one mailing address on their locator form during their first interview, and most subjects complied with this request regardless of their housing situation.

Randomization took place upon completion of the baseline interview. Study participants were randomly assigned to either the rechargeable incentive card (RIC) group or the money order group using a randomization list that was generated at the beginning of the study. Participants randomized to the RIC group received a rechargeable incentive debit card that allowed the research team to pay participants instantly by loading money on the card after the completion of a research activity. Those randomized to the money order group received the same amount of money as participants in the RIC group, but were paid with a money order following completion of each research activity. Money orders were provided to participants in person at baseline and sent via mail when monthly updates and follow-up interviews were performed. The randomization list was kept separate from a secure database containing contact information of the study participants. As such, the research assistants conducting tracking and locating activities for follow-up interviews were blind to the subjects’ group assignments.

RESULTS

We enrolled and randomized 151 subjects into the RIC condition and 152 into the control (money order) condition (N=303). As shown in Table 1, the groups were similar, with the one exception that RIC participants were slightly older.

Table 1.

Subject characteristics by study condition (randomized N=303)

Variable	RIC	Control	Total
Male	55.9	57.0	56.4
Age*	38.8 (SD=12.2)	35.7 (SD=11.4)	37.3 (SD=11.9)
Race/Ethnicity
-African American	11.2	12.6	11.9
-White-Non-Hispanic	56.6	50.0	53.8
-Hispanic	23.7	24.5	24.1
-Asian/Pacific Islander	2.0	2.7	2.3
-Other	6.6	9.3	7.9

^*p < .05

All participants were asked to call the UCLA research associate during the first 5 days of each calendar month for the ensuing 5 months in order to update their locator information—even if nothing had changed. Each call resulted in a $10 payment, either issued immediately via the RIC system or by money order within 1 to 3 weeks, by U.S. mail. The call-in rates by study condition are shown in Figure 1.

Figure 1.

Monthly call-in rates by study condition (N=303)

Rates in Figure 1 are provided for all study subjects and for those in outpatient treatment only. This is because the enrolled residential patients (N=60) did not act independently. Specifically, when one patient would access the shared telephone to call UCLA, s/he would then hand the phone to the other residential patients to update their information as well, undermining our ability to detect group-based differences. For all subjects, the only significant difference in call-in percentages was at Month 2. When we excluded residential patients, significant differences were found at Months 2, 3, and 4. Overall, RIC participants made 40% of the possible 5 calls, whereas control participants made 30%, a significant difference at the .05 level. Excluding residential patients, RIC participants made 39% of the possible calls, whereas control subjects made 27% (p < .01).

A secondary goal of this project was to assess the interaction between study condition and participants’ scores on the deferment of gratification scale. Our measure of Deferment of Gratification was a 12-item scale consisting of items such as, “Are you good at saving your money rather than spending it straight away?” Response options were 1 (no), 2 (neutral), and 3 (yes). Items were summed to produce an overall score of one’s ability to delay gratification. This scale has acceptable internal consistency (alpha=.72) (Ray & Najman, 1985). Scores ranged from 12–36, with a mean of 26.3 (SD=5.2) and a median of 26.

A 2 (study condition) × 2 (high versus low deferment of gratification) ANOVA did not reveal a significant interaction when predicting the total number of weekly calls initiated by participants (although pairwise contrasts did show significantly higher (p < .05) call rates among low-deferment RIC participants (Mean= 2.2, SD=1.9) versus low-deferment participants in the control condition (Mean=1.6, SD=1.8) or even high-deferment participants in the control condition (Mean = 1.5, SD=1.9). However, after dichotomizing the scores at the median, we compared the percentages of participants making at least half of the scheduled calls by study condition and deferment-of-gratification category (high vs. low). The RIC approach appeared to be uniquely effective for participants with low ability to defer gratification, whereas call-in rates among high-deferment subjects did not differ significantly by study condition. In fact, low-deferment subjects in the RIC condition were more than 60% more likely to make at least half of the scheduled calls than low-deferment participants in the control group (45.3% vs. 27.8%; χ² = .4.9; df = 1; p < .05).

Follow-up rates for the final interview that occurred 6 months post-randomization did not show a benefit for RIC. Excluding participants who were incarcerated, dead, or otherwise ineligible, we found final follow-up rates of 75% for RIC participants versus 79% of controls.

Client satisfaction

The follow-up interview was designed to assess participants’ satisfaction with the RIC method, identify logistical issues related to using the cards, and solicit input from the participants as to how the system might be further improved.

At the time of the 6-month follow-up, all but two of the RIC participants (98%) had activated their cards. When asked if they had experienced any difficulties, 92% reported that they had not, 6% indicated that they had, and 2% had not yet used their cards. Approximately one quarter of the RIC participants requested a replacement card during the 6 months after it was issued. Replacement cards were mailed within 7 days of notification.

To measure how participants perceived the RIC method relative to other potential payment methods, we asked those assigned to the RIC condition to rate several common methods on a 3-point scale, indicating that they were worse (1), the same (2), or better (3). RIC participants showed a decided preference for the RIC method over grocery cards (Mean = 2.9 [SD = .27]), money orders (Mean = 2.9 [SD = .35]), and points redeemable for goods or services (Mean = 2.8 [SD = .48]). Interestingly, there was even a slight, non-significant preference for the RIC method over cash payments (Mean = 2.3 [SD = .80]).

Time and cost analysis

For a period of 6 weeks, researchers observed field staff as they issued monthly incentive payments and attempted to locate study participants. Using a combination of observational data and contact logs, the cost team assigned values to each of these activities (using payroll records to calculate hourly rates). Once the costs of tracking and paying all of the study participants were determined, the data were disaggregated by study condition to reveal whether those assigned to the RIC condition did, in fact, require less staff time (and project costs) to track, locate, and pay.

Tracking and locating efforts were categorized into two domains: search functions and active engagement. Search functions involved any effort to find information on subjects from administrative data sources and search engines. These included: Department of Motor Vehicles lookups, Facebook lookups, jail lookups, prison lookups, and LexisNexis searches. Active engagement involved direct communication efforts. These included: letters to participants, e-mails to participants, and phone calls. In addition, time logs (with detailed time stamps) were kept for any activity involved in delivering payments to subjects and the actual cost of payment (price per money order and setup costs for debit cards). For subjects assigned to the RIC condition, the time spent on delivering online electronic transfers to RIC debit cards was recorded. For subjects assigned to the control condition, the cost team recorded the time involved with purchasing money orders and preparing the money order mailouts. For those cases where a single action applied to multiple cases, the time costs were distributed over the group of study subjects, to arrive at an average cost per participant. For example, when procuring money orders, a single trip to the post office would yield money orders for several subjects (this labor cost would be divided across the subjects to arrive at a per-subject cost).

Figure 2 shows the differences in search and location efforts, by group. We found only minor differences across groups. There were no statistically significant differences in the search functions performed. For direct engagements, we found a statistically significant difference in the number of e-mails sent and the number of phone calls made, but the magnitude of these differences was also small.

Figure 2. Average number of active engagements per participant (by group).

Note: *indicates statistical significance. The research team sent more emails to control subjects than to RIC subjects (p=0.07) and made more phone calls to control subjects than to RIC subjects (p=0.01).

There were staff time savings in tracking and locating, and significant staff time savings in executing payments for those in the RIC condition (staff time per subject was 34 minutes for RIC compared with 74 minutes for MO; a labor cost saving of $14 per subject).

The payment systems (RIC and money orders) differ in the per-participant cost of executing payments. Money orders have a flat fee of $1.25 per payment. The cost of issuing payment through a RIC system depends on the pricing structure used by card-issuing vendor. Vendors have a setup fee (this would enter a cost calculation as a fixed cost, which in most cases will vary by study size and also by the number of research studies underway in the contracting research institution) and a payment initiation fee per subject. The average cost per participant (the sum of the fixed and variable costs averaged over all subjects) therefore varies with study size, with cost advantages for larger studies performed at larger institutions (although lower-startup-fee pricing options are now available for smaller studies and for smaller research organizations).

DISCUSSION

The goal of this study was to assess the potential value of reloadable debit cards as a means of improving study engagement among hard-to-find research subjects. The study sample was drawn from participants in Los Angeles-based substance abuse programs, with one third describing themselves as homeless and at least 16% under probation/parole supervision. Given that reliable debit cards are the fastest growing payment instrument in the United States, with even greater penetration among persons who are unbanked and underbanked (Hayashi & Cuddy, 2014), the application of this approach for public health research involving itinerant populations seems promising.

Indeed, results from our randomized trial suggest there are several positive aspects to such an approach. RIC participants were more likely to initiate monthly update calls to the research center to update their locator information—this effect was particularly strong among participants with low ability to defer gratification. Our satisfaction survey indicated broad acceptance of the RIC approach, with scores showing a greater preference for the rechargeable debit card than for grocery cards, money orders, and redeemable points (and even a slight preference over cash payments). The amount of time devoted to tracking and paying participants was lower for RIC participants than for MO participants, with $14 in staff costs saved per subjects. Money orders cost $1.25 per payment. Whether a RIC payment system enjoys an overall cost advantage will depend on the front-end setup fee and the per-transaction payment fee of the card issuing vendor, with researchers at larger research institutions likely enjoying lower per-participant costs than their counterparts at smaller institutions.

Practice implications

For longitudinal public health research involving itinerant study participants, the RIC method produces a modest benefit in study engagement and reductions in staff time devoted to participant tracking and payments.