The Use of Technology in Participant Tracking and Study Retention: Lessons Learned from a Clinical Trials Network Study

Abstract

Background

The growing use of newer communication and internet technologies, even among low income and transient populations, require research staff to update their outreach strategies to ensure high follow-up and participant retention rates. This paper presents the views of research assistants on the use of cell phones and the internet to track participants in a multi-site randomized trial of substance use disorder treatment.

Methods

Pre-interview questionnaires exploring tracking and other study-related activities were collected from 21 research staff across the 10 participating US sites. Data were then used to construct a semi-structured interview guide which, in turn, was used to interview 12 of the same staff members. The questionnaires and interview data were entered in Atlas.ti and analyzed for emergent themes related to the use of technology for participant tracking purposes.

Results

Study staff reported that most participants had cell phones, despite having unstable physical addresses and landlines. The incoming call feature of most cell phones was useful for participants and research staff alike, and texting proved to have additional benefits. However, reliance on participants’ cell phones also proved problematic. Even homeless participants were found to have access to the internet through public libraries and could respond to study staff e-mails. Some study sites opened generic social media accounts, through which study staff sent private messages to participants. However, the Institutional Review Board (IRB) approval process for tracking participants using social media at some sites was prohibitively lengthy. Internet searches through Google, national paid databases, obituaries, and judiciary websites were also helpful tools.

Conclusions

Research staff perceive that cell phones, internet searches, and social networking sites were effective tools to achieve high follow-up rates in drug abuse research. Studies should incorporate cell phone, texting, and social network website information on locator forms; obtain IRB approval for contacting participants using social networking websites; and include web searches, texting, and the use of social media in staff training as standard operating procedures.

INTRODUCTION

Successful participant retention is essential in randomized clinical trials (RCTs).^1–3 Loss to follow-up can threaten internal validity by introducing bias in estimating differences between treatments, as participants lost to follow-up may have different outcomes than participants who were interviewed.^4,5 Furthermore, attrition can result in a loss of statistical power.⁶

A number of approaches have been reported to help research staff find and successfully communicate with, track, and interview participants, thereby optimizing study retention.⁷ Some of these approaches include providing financial incentives for participants^3,8; making repeated contacts with participants through phone calls, reminder postcards, letters, and contacts with designated family members or others^8–10; flexibility in scheduling interviews^1,11; and home visits.⁸ A report of participant perspectives regarding factors that increased study retention found that rapport with research staff, relatively short interviews, and participant belief that the study provided them with useful information led to high follow-up rates.¹

Tracking and communicating with participants enrolled in substance use disorder (SUD) research is especially challenging. There have been several reports of approaches to successful completion of long-term research follow-up, including successful strategies utilized within individual longitudinal studies^2,12,13 as well as a summary of lessons learned across numerous research experiences. Participants in SUD research are often difficult to reach because of the chaotic lifestyle resulting from their addiction. They may change residence and phone numbers frequently; they may be incarcerated or hospitalized; or, in some cases, they may be evading the authorities. To minimize study attrition, it has been noted that having and updating detailed locator information and having a familiarity with field tracking techniques are of great importance.^2,14 Scott⁷ reported on a successful model to achieve over 90% follow-up rates that included a structured follow-up protocol focused on participant engagement and verification, maintenance, and confirmation of contact information prior to the scheduled follow-up visit.

While the follow-up techniques described above were successful prior to the explosion of internet search capability, social media, and the use of smart phones and text messaging, they are still highly relevant. However, the expansion of newer technology has added both complexity and opportunity to follow-up field work for clinical trials. Studies as diverse as health survey research¹⁵ and sexually transmitted disease treatment research³ have reported on the use of texting for tracking and maintaining communication with participants, as well as the use of internet searches for tracking.¹¹ In an urban health study, Schneider and colleagues (2011)¹⁵ compared access to digital communication technologies and its association with health status and study retention. They found that 84% of study participants had a cell phone and 62% had experience using their cell phones for texting. Study participants that had experience texting or who had used the internet for five or more years were significantly more likely to have been located for their three-month follow-up. Despite the seeming ease of maintaining contact through texting and other newer technologies, not all individuals would wish to be contacted through these approaches. In a survey of attitudes toward the use of cell phones in healthcare delivery, Person et al. (2011)¹⁶ found that nearly 30% of survey participants did not believe that text message reminders were acceptable because of cost, confidentiality concerns, or not wanting to be bothered. However, the negative aspects identified by participants may be off-set by the benefits to both participants and staff, and further examination of their utility is warranted, given the rapidly changing social norms concerning the use and acceptance of such technologies.

The purpose of the present study was to utilize a structured qualitative methodological approach to examine the use of technology for participant tracking and communication from the perspective of research staff conducting a multi-site study through the National Institute on Drug Abuse Clinical Trials Network (NIDA CTN). Previous researchers have examined quantitative outcomes based on methods of participant contact¹⁷ or have used a less structured approach to gather descriptive data than was utilized in the present study.¹¹ By focusing on the use of newer technologies to track and communicate with participants in the parent study, this paper seeks to add to the extant literature on this topic.

METHODS

Parent Study

The parent study, Web-delivery of Evidence-Based, Psychosocial Treatment for Substance Use Disorders (WEB-TX; CTN-0044), evaluated the effectiveness of a web-based version of the Community Reinforcement Approach intervention plus incentives targeting drug abstinence and treatment participation for 507 adult men and women starting outpatient substance abuse treatment at one of the ten participating clinics.^18,19 The WEB-TX study sites were selected in order to obtain a broad range of geographic, racial and ethnic diversity and the overall sample was 38% female, 53% White, 22% African American, 11% Hispanic/Latino, and 14% multiracial/other. The mean age was 35 years (SD=10.9), 41% were employed, and 35% indicated that their treatment entry was mandated by the criminal justice system. Primary substance use reported by participants at baseline was: cocaine (20.32%); marijuana (21.10%); opiates (21.30%); stimulants (14.00%); alcohol (21.10%); all other (2.17%). Follow-up rates of 89% and 91% were achieved across all sites at three months and six months, respectively. WEB-TX received Institutional Review Board (IRB) approval and all study-related activities, including tracking, were conducted in accord with human subject protection. The present study was reviewed and determined to be exempt by the Friends Research Institute’s Institutional Review Board.

Participants

Pre-interview Questionnaire Participants

One questionnaire was sent to each of the 10 WEB-TX study sites, which were located throughout the US (see Campbell et al., 2013 for details).¹⁹ A total of 21 research staff members completed this pre-interview questionnaire, with all but two sites completing the questionnaire responses as a team. Respondents included two Site Principal Investigators (PI), one Co-PI, nine Research Coordinators, two Project Managers, six Research Assistants (RAs), and one Quality Assurance Monitor. All but two of the questionnaire respondents were female and all were adults. Responses were then used to develop the semi-structured interview guide.

Semi-structured Interview Participants

Interviews were completed with 12 of the research staff members who responded to the questionnaire for their site. All 12 staff members were female, and included 8 Research Coordinators, three of the Research Assistants, and one Site PI. All were highly involved with and knowledgeable about participant tracking activities at their site. All but one site chose to have a sole respondent participate in the interview. The 10^th site chose to have their three most involved team members complete the interview as a group.

Pre-interview Questionnaire

In order to develop the semi-structured interview guide, a pre-interview questionnaire was first administered to better determine areas pertinent to participant retention. The questionnaire included eight open-ended questions regarding: study culture and its influence on participant tracking and retention; the use of locator forms; successful follow-up tactics; the influence of local factors on tracking and retention; characteristics and tracking of especially challenging participants; what to do when the locator forms’ utility had been exhausted; and suggestions for future RCTs.

Semi-structured Interview

Responses from the questionnaires were examined and used to construct the semi-structured interview guide, which consisted of the following five topic areas: the use of technology in tracking; issues of coordination between research and clinical staff; techniques for building rapport with study participants; techniques for tracking particularly difficult to reach participants; and indicators of a participant’s life instability. All interviews covered these five topic areas and follow-up questions were asked in order to elicit greater detail. The 10 interviews with the 12 participants were conducted by phone by the first author, a researcher with 15 years’ experience conducting qualitative research. Interviews ranged in length from 30 to 60 minutes. All participants were provided with an information sheet in lieu of written consent and were not paid additionally for their time, as interviews were conducted during normal working hours.

Qualitative Analysis

Completed pre-interview questionnaires and typed phone interviews were entered into Atlas.ti for analysis.²⁰ Grounded theory methodology, a qualitative research approach that systematically analyzes data and inductively builds theory, was used in our analysis.²¹ During the open coding phase, the investigators approached the data looking for descriptions of any type of technology used for tracking, whether in response to the specific “technology question” or any other data from the surveys and interviews, and determined there were three main types of technology use described: cell phones for calling, cell phones for texting, and various internet functions. During the selective coding phase, the data were categorized into “use by the study participant” and “use by study staff” sub-codes, and then further categorized in terms of “facilitators to tracking” and “barriers to tracking.”

RESULTS

Cell phones for calling

One of the most frequently mentioned tools used successfully by the RAs was the cell phone, in both expected and unexpected ways. Many study participants led chaotic, somewhat transient lives. While their physical address and home phone numbers were subject to change, even those participants who relocated possessed a cell phone, which was essential for maintaining contact. However, cell phones were not always reliable. Study staff mentioned that participants’ cell phones were often disconnected because their minutes were depleted on pay-as-you-go plans. Sometimes cell phones were lost or stolen. Relying on a participant’s cell phone as the primary means of contacting participants could prove frustrating when its service was discontinued. However, several study staff continued to call the non-working number because service was frequently re-activated once the participant obtained the necessary funds.

For study participants, cell phones also had an advantage over traditional phones in that they captured incoming phone numbers, thereby providing call-back numbers for staff. The call-back feature also minimized the need for participants to retain study phone numbers and eliminated the risk of writing call-back numbers down incorrectly. Some of the staff likewise benefitted from the “incoming number capture” feature of cell phones as they would write down the incoming phone number and compare it to those on the locator form so that the locator could be updated.

Using study cell phones also permitted the RAs to call participants (and/or receive calls) during evenings or weekends when they were out of the office. In addition, for confidentiality reasons, the clinic landlines sometimes displayed as “unknown number” on the receiving end and some participants would not answer calls from unknown numbers. The staff believed that having a study cell phone provided a number that was recognizable to participants, increasing the likelihood that participants would answer the call. An extra benefit to both participants and study staff was that participants who were having difficulty or had dropped out of treatment were not required to call through the clinic in order to reach the research staff..

While many study participants possessed a cell phone, research staff reported that paying for the phone was clearly challenging for participants. Participants who lost their phones or who used “disposable” phones and changed them frequently often did not keep the same phone number, so updating locator information at each study visit was crucial. Additionally, participants sometimes chose not to answer a call, check messages, or voice mail boxes were at capacity and research staff was unable to leave a message. Also, staff had to be aware of confidentiality issues when calling because at times participants lent their phones to others.

Cell phones for texting

A surprising finding for research staff was that participants treated and responded to cell phones differently with respect to texting than they did to calling. This finding was important to note because, while some sites explicitly asked participants if they would be willing to receive study-related texts, many locator forms did not initially include this permission and had to be amended. Staff indicated that some participants would respond with a follow-up text if a voice message had been left on their cell phone. When this occurred staff would assume that the participant was amenable to being texted, would note it on the locator form, and use texting for future communications.

Texting appeared to be a particularly effective way of communicating with younger adults. Use of texting was helpful, and often necessary, because staff reported that younger participants were also more likely to drop out of treatment early and proved to be some of the most challenging participants to track.

Staff reported that text messaging had certain advantages compared to voicemail. For example, one staff member noted that participants often looked at texts, even if they intended to delete them. Texts are succinct and the first few words of the text can be viewed without even opening the full message. Thus, numerous respondents mentioned that they believed text messages were more likely to be received and responded to by the participant than voicemail messages. This aspect of texting also benefitted research staff, who reported that they sent texts using study cell phones or free online accounts with numbers which participants would not easily recognize. Although some participants would not answer or return calls from unknown numbers, they were more likely to read the text, recognize that it was from the staff, and respond. Texts also left a written record of the message that could be referenced later, so the participant did not have to transfer the message to writing, as is customarily done with voicemail or phone calls. Staff reported that they texted appointment reminders and included crucial information such as date, time and location of the appointment.

Finally, the brief, casual construction of texts and their use with more familiar acquaintances seemed to enhance the relationship between staff and participants. It was almost as if permitting text messaging, even if done so only for the sake of expediency, created a different kind of relationship and, as one staff remarked, it helped to build rapport.

The internet

As previous studies have shown,²² even populations with few resources, such as homeless persons, have access to computers and the internet. Our study team noted that public libraries often served two important purposes for their participants, some of whom were homeless: as a place to access computers and as a place to avoid extreme weather conditions.

For the many participants who did not have their own computers, accessing the internet meant using the library to check their free email accounts or access Facebook. In fact, internet-related communications were often the most economical, if not the most efficient, way of communicating with some participants. But while participants had barriers to receiving email communications, some staff also had barriers to sending internet-related communications. Study teams working within hospital systems were more likely to have limitations regarding electronic communications, in general, and internet communications, specifically, with some unable to send communications to participants via the hospital’s email system.

Confidentiality concerns were often behind such limitations because emails leave a potentially traceable record of study participation. Similarly, confidentiality concerns were a major obstacle to some sites being able to use social media messaging options. The sites that used Facebook opened a generic account and did not include any actual personal staff information or link it with the study in any way. The study teams never posted public messages on participants’ Facebook pages, and many had a policy of not “friending” participants, but instead would send them a private message (similar to an email message and only visible to the recipient) containing minimal information. While non-friend messages typically are not sent to a person’s regular “Inbox” on Facebook, for a nominal fee ($1 US) private messages could be initiated from the study team’s anonymous account and sent to the participant’s regular inbox, which then notified the account holder that a new message had been received, and also permitted the study team to see when the message had been viewed.

Staff mentioned that some people were far more likely to update their Facebook pages and check message postings than respond to email messages. However, according to staff, obtaining IRB approval to use social media for contacting study participants was a long process for sites that chose to pursue this option. Even those sites that could not contact participants directly through social media were sometimes able to search for participants online, including accessing participant information using sites such as Facebook or Myspace when participants made their profiles public.

Many websites can also be used to send text messages, even without a cell phone. The research teams that did not have a dedicated study cell phone also noted that they would use this type of platform (e.g., Google Voice) to send text messages to participants’ cell phones.

Using the internet for searching was far more common than using it to communicate with participants. All sites mentioned using search tools and various online searching strategies, and most sites mentioned multiple search tools for locating participants, such as Google, Lexus Nexus, national paid databases, obituaries, and legal sites.

Research staff also reported that Google maps was used for a variety of purposes: to verify addresses when filling out locator forms; for finding homeless shelters or potential work addresses; to get directions for home visits; and even as a tool for meeting location options for off-site follow-up interviews.

Legal/judiciary websites such as Victim Information and Notification Everyday (vinelink.com) were particularly helpful for checking on missing participants because many times “missing” participants had actually been arrested. These publicly accessible local sites not only helped the study team confirm an incarceration, but some judiciary sites also sent an email notification when a person of interest was being released. Because research teams were not able to interview incarcerated participants, this later information was extremely helpful, as it informed staff when they should resume attempts to contact the participant.

DISCUSSION

Effective participant tracking and communication to help achieve high follow-up interview rates are critical to the success of randomized clinical trials.^1–6 With the growth in use of newer search and communication technologies, even among low income and transient communities,²² a research staff’s ability to use these technologies can create additional opportunities to track and communicate with participants. The present study extends the literature on study retention strategies by examining research staff’s perspectives on the use of cell phones, internet, and social media technology to track and communicate with participants, and ultimately achieve a high follow-up interview rate in drug abuse research.

Cell phones were particularly useful for facilitating participants’ study retention for a number of reasons. First, cell phone ownership was common and these devices were repeatedly identified by staff as a primary means for communication with study participants. Second, the value of cell phones in facilitating contact with participants did not appear to be limited to those phones owned or carried by the participants. Third, cell phones allowed for multiple ways to communicate with participants, including text messaging. Indeed, in several instances, texting was believed by study staff to be superior to phone calls for communicating with participants and scheduling follow-up interviews. This finding is consistent with a study by Maher and colleagues¹⁷ in which approximately three-quarters of 48 methamphetamine injection drug users participating in a trial of HIV-risk reduction indicated that they could be reached by text messaging and at one point in the study texts were the only modality for contacting six percent of their sample.

Our research staff reported using websites to look up phone numbers and addresses to verify the validity of locator information. In addition, the internet was a valuable tool for locating participants who were missing. We found that the ability to track and, many times, reengage missing study participants was maximized by using the internet to search legal databases, identify the addresses of local homeless shelters, and places of employment. Finally, the internet also provided mechanisms for contacting both active and difficult-to-reach participants. Much of the aforementioned success with text messaging was conducted using internet accounts created specifically to send text messages.

As with cell phone calling and texting, the internet was not a fool-proof tool, and all research teams stressed that the best way to track participants was to use a variety of approaches, including but not limited to, the range of technological tools available to maintain or resume contact with participants. They noted that finding the “best” way to communicate with each participant was something that should be identified early in the conduct of a study, but that flexibility and persistence were the best tools for finding participants and achieving a high follow-up rate.

One less well-studied way to locate and contact research participants via the internet was the use of social networking sites such as Facebook. More than two-thirds (69%) of American adult internet users use at least one social networking site.²³ In a longitudinal study of at-risk families, Mychasiuk and Benzies²⁴ reported locating 19 participants using Facebook, decreasing their study attrition by 16%. Facebook was also used to contact 9% of participants in a longitudinal study of adults who used methamphetamine.²⁵ Our findings were consistent with these studies in that social networking, and Facebook in particular, was identified by research staff from multiple study sites as an additional tool used to track and contact study participants. Further, we found that even when IRB permission to contact participants using a social networking site had not been granted, sites such as Facebook were useful in gleaning potential contact information such as alternative email addresses and phone numbers.

Though these newer technologies have many benefits, they are not without their challenges. The lives of persons with SUD are often marked by instability and impermanence. Staff in our study found that participants were many times unable to be reached by cell phone due to the lack of calling minutes on the phones, frequent changing of numbers, and the fact that cell phones were sometimes lost, stolen, or confiscated. Internet access was often inconsistent and email accounts were infrequently accessed, particularly by those participants who were homeless or more transient.

Technologies also came with additional privacy and confidentiality issues. While texts, emails and messages sent through social networking sites had the benefit of allowing the participant to save the message for later viewing, they also left a written record of the contact. Some IRBs may be hesitant to allow use of social networking sites for contacting participants due to frequently changing privacy settings. Additionally, these messages may be less effective for participants with low literacy. Participants may also be unwilling to provide comprehensive information on locator forms during their initial study visit because of symptoms related to their substance use or mental health disorders. Research staff should respect participants’ privacy, build rapport and attempt to increase locator information over time in studies that involve multiple research contacts.

Based on our findings, we offer several suggestions for future studies. When collecting locator information, research staff should take care to collect comprehensive information including cell phone numbers, email addresses, and the profile name and web address for social networking sites. This information should be collected not only for the participant, but, if known, for all contact persons listed on the locator form. When possible, we advise research staff to use the internet to verify the accuracy of the information provided at the time it is collected. For participants with cell phones, we recommend that research staff inquire about the following: 1) ability and willingness to receive text messages; 2) times of the month that participants are likely to deplete their cell phone minutes; and 3) frequency of changing phone numbers. Researchers should consider including a dedicated cell phone number or numbers for research staff and may also want to consider purchasing cell phones or cell phone minutes for participants in their budget projections. When contacting participants using these communication technologies, research staff should be aware of the associated confidentiality concerns and have procedures for minimizing risk of breeches. Text, email and social networking messages should be generic and include only the minimum amount of information necessary. With respect to social networking sites, researchers should avoid posting messages on participants’ profiles and instead use the private messaging feature, which is only visible to the recipient. When using social networking sites to search for missing participants, we recommend confirming a participant’s identity by cross-referencing information such as email address(es), birthday, and friends with previously collected locator information as described by Bolanos et al.²⁵ Finally, we recommend that these technologies be used in addition to, rather than in place of, established study retention procedures as part of a comprehensive, multimodality tracking protocol.

Limitations

Our findings should be interpreted in the context of the following limitations. Interviews were only conducted with research staff from a single study. Though participants were drawn from one RCT, it was a multi-site trial including ten geographically diversely located sites throughout the continental USA and Hawaii, which increases the generalizability of our results. The size of our sample was small and there was redundancy in the data, such that the pre-interview surveys used to construct the interview guide were also used as data in our analyses. As the larger WEB-TX study focused on a web-based treatment approach, it is possible that the clinic patients who enrolled in the parent study were those who were the more tech-savvy than the others. However, the use of smart phone technology is rapidly increasing throughout the world and thus, despite this limitation, our study findings should be of value to researchers. Naturally, these findings related only to studies in which data are gathered from participant interviews and do not apply to other sorts of data gathering. Finally, our findings regarding the utility of newer search and communication technologies in tracking and retaining study participants come from interviews of research staff. Thus, we are unable to comment on study participants’ receptivity to use of these technologies.

Conclusions

Despite these limitations, our findings suggest that research staff perceive cell phones, internet, and social networking technologies to be effective tools in achieving high retention rates in drug abuse research but that staff persistence was paramount. Future research should systematically examine participant preferences regarding contact by research staff and glean additional information regarding ways to enhance study retention. Finally, being able to effectively track and contact study participants for follow-up interviews is just one step in retaining them in clinical trials. Technology should be viewed as an additional tool for maintaining contact and developing rapport with study participants.