Optimizing follow-up and study retention in the 21st century: Advances from the front line in alcohol and tobacco research

Abstract

Aims

Longitudinal studies are integral in addiction research but retention of participants over time can be challenging. While statistical algorithms for missing data have advanced, they remain less desirable than collecting actual data with high retention rates. An update to methodological primers with consideration of evolving technology and privacy concerns is needed for 21st century researchers.

Methods

Comprehensive follow-up methodological strategies were conducted in four concurrent laboratory-and intervention-based studies across N = 697 drinker and smokers enrolled in studies at the Clinical Addictions Research Laboratory at the University of Chicago. The methods of three key longitudinal research themes and their outcomes are outlined, including: a) mindset of the research team starting at study enrollment, b) modalities with a particular focus on advances in technological strategies in follow-up, and c) mitigating difficult to reach and challenging participants.

Results

The techniques described herein produced follow-up rates of 95% and 99% in two laboratory-based studies with follow-ups of 1- and 6-years, respectively and 94% and 97% in two intervention studies with follow-ups of 6- and 12- months. Adapting incentive strategies more than tripled on-time follow-up, from 18% to 68% of the sample, switching to more advanced technologies decreased participant burden and time by 30% from traditional telephone interviews, and difficult-to-reach participants averaged 47 contact attempts.

Conclusions

The methods presented produced exceptional follow-up retention across four studies. The principles and methodologies discussed may be modified across a range of studies to target various sub-populations in the addiction field.

1. Introduction

Alcohol and tobacco use remain two of the top three leading causes of preventable disease worldwide (Bauer et al., 2014; World Health Organization, 2010). The preponderance of studies examining these and other addictions has historically been cross-sectional [e.g., National Epidemiologic Survey on Alcohol and Related Conditions (Hasin and Grant, 2015), Monitoring the Future (Johnston et al., 2016), National Youth Tobacco Survey (Centers for Disease Control and Prevention, 2015)], but more longitudinal studies and/or subsequent analysis of existing datasets are needed to elucidate cause-and-effect factors (Maslowsky et al., 2015), developmental course over time (Resnick et al., 1997), cohort effects (Bacharach et al., 2007), and treatment and disease-related outcomes (Buccheri et al., 1996). Clinical researchers, particularly those in the addiction field, are often reluctant to embark on long-term studies given concerns about retention rates, restricted funding cycles, and project staff attrition (Robinson et al., 2005; Streissguth and Guinta, 1992). At the same time, there are examples of successful community cohort studies (Vaillant, 2003), outpatient follow-up studies (Scott, 2004), and laboratory studies with longitudinal follow-up (Schuckit and Smith, 1996).

In the aforementioned studies and in biomedical and psychosocial research in general, participant retention can be a challenge. Follow-up rates reported in published studies are often in the low-to-moderate range, from under half up to three-quarters of the original sample (Booker et al., 2011; Hansen et al., 1990). Overall follow-up rates are considered good at 70% (Mangione, 1995; Scott, 2004) when utilizing traditional methods such as mail-in surveys, telephone interviews, and in-person assessments. Careful attention to these follow-up rates are important as decreased sample size may lead to distorted accuracy of results and study bias as the sample may no longer be representative. More specifically, if the study’s accuracy is compromised, conclusions made by the researcher may be erroneous and may negatively impact the internal and external validity (Barry, 2005). To account for missing data, investigators may employ advanced statistical techniques (Cotter et al., 2002; Wood et al., 2004). However, such statistical imputation techniques only provide a “best guess” as they require assumption of missing mechanisms (missing completely at random or missing at random; Gray, 2016). Unforeseen difficulties may arise when such algorithms are used in datasets with high proportions of missing data, large numbers of variables, small sample sizes, and data not missing at random (Sterne et al., 2009). Thus, establishing and maintaining the highest possible retention in longitudinal studies is crucial for data accuracy and to avoid pitfalls of imputation that may lead to statistical bias and affect data interpretation.

A meta-analysis of 85 longitudinally followed cohorts published in 1990 (Hansen et al., 1990) pointed to the need for more papers on retention and tracking methods specifically in addictions research. Shortly thereafter, Schuckit and colleagues (Twitchell et al., 1992) described the follow-up methodology of their San Diego Prospective Study, a laboratory and follow-up study of primarily young Caucasian male college drinkers with an impressive 99% follow-up rate (450 out of 453) during the first follow-up, 8–12 years after enrollment (Schuckit and Smith, 1996). While this 1992 primer described the architecture of excellent follow-up retention, it is now over two decades old. More recently, in 2004, the Engagement, Verification, Maintenance and Confirmation (EVMC) model was reported for follow-up in outpatient addiction treatment samples from established programs (Scott, 2004). Follow-up rates of at least 90% were cited and methodology was described for repeated contact attempts. Retention strategies have also been outlined in other work published in the 1990s and 2000s (Booker et al., 2011; BootsMiller et al., 1998; Desmond et al., 1995; Wutzke et al., 2000), but these pertain to retention methods used before the recent technological epoch that has largely changed the way that people communicate.

Based on our own successful retention over four recent and concurrent studies in drinker and smoker samples, herein we provide an updated framework for researchers to consider in conducting longitudinal research in the 21st century. Our work spans from laboratory-based to pharmacological and behavioral treatment studies across four datasets in 697 total participants. Across these studies conducted at the Clinical Addictions Research Laboratory at the University of Chicago, the samples ranged from small (n = 30) to large (n = 290), the number of follow-up assessments after enrollment ranged from 1 to 11, and the published follow-up intervals ranged from 1 month to 6 years (and continuing). Up to this juncture, we have not been able to fully describe the follow-up techniques due to publishing constraints on methodological detail in main outcome papers.

2. Methods

The four concurrent longitudinal studies using the methodology described in this paper include two laboratory-based cohort studies, the Chicago Social Drinking Project (CSDP; King et al., 2011; King et al., 2014; King, 2016) and the Emerging Adult Smoker Study (EASS; Conrad et al., 2013), and two intervention trials, the Chicago STOP Smoking Research Project (CSTOP; King et al., 2012) and the Chicago Young Adult Health Study (CYAHS; Fridberg et al., 2015). Each study’s design and purpose, sample characteristics, and follow-up procedures are included in Table 1. Follow-up for each study was based on the study’s purpose, including, but not limited to: drinking, smoking, drug use quantity and frequency, timeline follow-back calendars of past month daily use estimates, diagnostic symptoms, consequences, life transitions, affective status, adverse events, and other health outcomes. Retention methods are summarized in three main areas including: a) mindset, starting at enrollment and including all members of the research team, b) modalities, with particular consideration of technological advances, and c) mitigating difficult to reach and challenging participants.

Table 1.

Study Design, populations, and follow-up.

Study Design and Purpose	Chicago Social Drinking Project (CSDP)	Emerging Adult Smoker Study (EASS)	Chicago STOP Smoking Research Project (CSTOP)	Chicago Young Adult Health Study (CYAHS)
Study Type	Lab Study with Follow-Up	Lab Study with Follow-Up	Clinical Trial	Clinical Trial
Enrolled	2004–2006; 2009–2011	2006–2007	2006–2009	2011–2014
Sample	Heavy and light non-dependent social drinking young adults	Adolescent and young adult experimental smokers	Adult nicotine dependent smokers	Young adult heavy-drinking smokers
Main Inclusion Characteristics	Drink 1–50 drinks/week	Smoke 1–20 cigarettes/ week	Smoke 70–280 cigarettes/ week	Drink 5–40 drinks/week; Smoke 4–132 cigarettes/week
Purpose	Examine relationship of alcohol response to future drinking behavior	Examine relationship of lab-assessed stress response to future smoking behavior	Determine efficacy of 12 weeks of naltrexone vs. placebo on smoking cessation program outcomes	Assess effects of alcohol response brief intervention to future drinking and smoking behavior
Sample Characteristics
Sample Size	N =294	N =56	N = 315	N= 32
Age (Years, Mean ± SD)	25.4 ± 3.0	20.8 ± 2.2	41.9 ± 11.3	23.8 ± 2.1
Sex (Male)	58%	59%	47%	56%
Education (Years, Mean ± SD)	15.9 ± 1.7	78% some college; 22% college graduate	15.1 ± 2.3	15.0 ± 1.8
Race (Caucasian)	77%	70%	57%	53%
Follow-Up Design and Success
Assessment Intervals	Brief quarterly through 24-months, more extensive annually (12-, 24-, 48-, 60-, 72-, 96-, 120-months, and ongoing) post lab sessions	12-months post lab session	6- and 12-months post smoking quit date	1- and 6-months post intervention sessions
Primary Modes	IVR, Telephone Interview, Online Surveys, Mailed Surveys	Online Survey	Telephone Interview, In-Person Assessment	Online Surveys
Follow-Up Incentives	$10 for 5-min. quarterly, $40 for 20–30 min annual. Increased compensation (40–50%) for on-time completion with entry into $100 annual raffle. Payment increased to $60 and $100 at 60- and 120-months, respectively	$10 for 10–15 min survey, entry into $50 bi-annual raffle for successful completion	$30 for 15–20 min follow-up, 1 in 4 chance drawing for $30 or $60 for in-person smoking cessation verification, entry into $50 annual raffle for on-time completion	$10 for 10–15 min follow-up, 50% increased compensation for on-time completion
Success Rates (completed/possible)	120-months: 3431/3468 (99%)	12-months: 53/56 (95%)	6-months: 236/239 (99%); 12-months: 231/239 (97%)	1-month: 31/32 (97%); 6-months: 30/32 (94%)

2.1. Mindset

Early in a study, fostering a mindset of regular communications, positive alliance, and study identity have all been described as important elements to increase participant retention (BootsMiller et al., 1998; Desmond et al., 1995; Twitchell et al., 1992; Wutzke et al., 2000). We implemented these elements across our studies, so that during screening, participants are well informed of the frequency, modality, and expectations for follow-up, including the importance of retention for the scientific rigor of the study. Study candidates were not enrolled if they were unable or unwilling to provide contact information for themselves, or for collateral persons in the event that they could not be reached (see Table 2 for more details). Of note, disadvantaged subgroups, such as those who are severely addicted, medically compromised, or homeless have also been shown to be able to provide some contact information (Bonevski et al., 2014). Specific elements of our mindset are described in this section and include building a positive alliance, fostering study identification, creating and maintaining a participant-centered study website, and adopting an overall team mindset, with all members of the research group involved in follow-up.

Table 2.

Methods for Participant Contact and Study Customization.

Contact Informationa
Participant Information	Legal names, preferred name, nicknames, maiden names; current and permanent addresses; personal and work e-mails; cell, home, and work phone numbers; usernames/handles for social media sites (Facebook, Twitter, Instagram, LinkedIn, blogs), etc.
Identifying Information	Date of birth, Social Security number, license or ID card numbers, etc.
Work and Affiliations	Work contact and occupation, school and major, group associations, social clubs, etc.
Collateral Information	At least 2 people (family members, spouse, partner, close friends); Names, addresses, phone numbers, e-mails, etc.
Study Customization
Branding	Unique and eye-catching study name and logo; include branding on mailings, websites, gifts, etc.
Academic Detailing	Customized to study population; includes study logo and contact info; items that participants could use regularly or store (pens, key chains, etc.)
Study Website	Participant-centered and specific to the study, online links to surveys, study updates, staff contact information, directions to lab, etc.
Newsletters and Cards	Friendly, creative photos and interviews with staff, study updates, reviews of related-research publications, engaging puzzles, catchy titles, cards signed by hand for added personalization
Data Collection Adaptations	Multiple versions of surveys (paper, telephone, online); relevant communication methods to study population and prevailing technology (i.e. text messaging, emails, social media)
Compensation and Incentives	Consider sample, time spent, cost of living, study duration and total effort for compensation (i.e. cash, gift cards, gifts, vouchers, etc.); increase amount for on-time completion, extended participation, and include additional raffles
Staff Incentives	Group lunches, performance recognition, monetary rewards, etc.

^aReferences to information found in the table include: Bootsmiller et al., 1998; Cottler et al., 1996; Hunt and White, 1998; Kleschinsky et al., 2009; Scott, 2004; Twitchell et al., 1992.

2.1.1. Alliance

Formation of a positive alliance is a cost-effective, crucial element in longitudinal studies (DeWitt and Brady, 2003). Our group applied this principle across all of our studies by carefully training research staff to form a mutually beneficial and positive relationship with participants (Bruning, 2002). In staff trainings, we employed a series of certification requirements for conducting study interviews with videotape review to assure both standardized communications and friendly/engaging interactions. In addition, our study staff regularly recorded participant’s occupation, hobbies, and interests in the participant’s confidential file in order to foster re-contact attempts over time and to create a personalized staff-participant relationship.

2.1.2. Study identification

Fostering study identification is often advised in follow-up procedures (Hunt and White, 1998; Nicholson et al., 2011), with the goal for participants to gain a sense of familiarity and “ownership” of their role within the research project. In our work, identification began with creation of a branded name and logo for each study (see examples in Fig. 1). These were included in recruitment notices, letterheads, cards, newsletters, and websites. The only exception to this procedure related to communications with collateral persons, which were provided by the participant. In such contacts with collaterals, only the phrase, “the study at the University of Chicago” was utilized in order to ensure participant confidentiality.

Fig. 1.

Branded name and logos created for each study.

Study identification was also facilitated by academic detailing with gift items given to participants at various stages of participation and customized to the sociodemographic characteristics of each sample. These items were able to facilitate retention by including contact study information on each gift item, which ranged in cost depending on each study’s budget and scope. For example, inexpensive matchbooks with the study logo were disbursed at bars and local social events during recruitment efforts for the EASS, which had a modest budget. In CSTOP, stress balls were given as a coping tool on the quit date as well as glow-in-the-dark magnets with tabs as reminders of follow-up dates. Finally, for the CSDP, which has included more extensive and longer-term follow-up of light and heavy drinkers over time, participants were given gift items that may be regularly stored around one’s home, i.e., bottle openers, pens, flashlights, and reusable shopping bags.

2.1.3. Participant-Centered study website

For each study, new websites or links within the main laboratory website were created as a point of contact for interested candidates as well as for enrolled participants. In contrast to many laboratory websites that are framed for academic colleagues and the research community, these websites were “participant-centered.” As such, these user-friendly sites included easy drop-down menus to access study information, procedures, and follow-up schedules. They also included useful “go to” resources for study announcements, links to online follow-up surveys, and contact information for study staff. With the central role that the internet now plays for obtaining information (File and Ryan, 2014), participant-centered websites represent a particularly important manifestation of a research team’s professionalism and mindset.

2.1.4. A team mindset: it takes a village

A team-based approach is important in establishing a cohesive longitudinal research framework (BootsMiller et al., 1998; Vincent et al., 2012). This took shape through weekly lab meetings that included updates on follow-up activities, challenges, and problem solving. This strategy helped to reduce staff burnout and frustration particularly in the case of difficult-to-reach participants (see section 2.3). Additionally, staff incentives have been suggested to enhance motivation (e.g., BootsMiller et al., 1998); as such we also implemented social rewards to reinforce success in follow-up and to increase staff cohesion via team-building honors. For example, an “eye of the tiger” award was given to a staff member for outstanding efforts and complimentary lunches were provided to celebrate the team meeting particular goals related to study follow-up. Finally, successful follow-up was conceptualized as both a top-down and bottom-up approach that incorporates all staff, including the principal investigator (PI; Piccolo and Colquitt, 2006). The PI can be invaluable in modeling professional communication and perseverance (Taylor et al., 2005) and can empower staff to improve team effectiveness (Burke et al., 2006). In all of our studies, the PI oversaw all follow-up strategies on a weekly or monthly basis and participated directly in follow-up for hard-to-reach participants (see section 2.3).

In sum, a mindset of regular communications, positivity, and study ownership is important for both participants and the research team. In the midst of emerging technology, this ethos is complimented by newer data collection modalities, as described in the next section.

2.2. Modalities: including 21st century technologies in follow-up

In the current digital age, it is becoming increasingly important to leverage today’s progressively technology-centered society for the collection of longitudinal data and participant communication. Historically, the primary modalities to contact and communicate with research participants have reflected the prevailing methods of the time, i.e., mail, telephone contact, and face-to-face assessments. Newer technological methods, however, are continually being developed to reduce cost and increase convenience. Accordingly, when used in a research setting, these emerging technologies may reduce study cost, staff burden, and increase participant convenience. Furthermore, in our increasingly mobile society (U.S. Census Bureau, 2007), a more nationally representative sample can now be reached via web and mobile mediums as age, class, gender, and racial differences in internet usage have decreased (File and Ryan, 2014). In this section, we build upon established methods that have been estimated to increase participant retention by 3–24% (Booker et al., 2011) (see Table 2).

2.2.1. Interactive voice recording (IVR)

Interactive voice recording (IVR) systems allow surveys previously administered via phone or face-to-face interviews to be automated, with participants using keypad entry to respond to computerized voice prompts and survey questions. IVR has been shown to yield data similar to face-to-face interviews with more flexibility in terms of when and where surveys are completed, which decreases workload for the research team (Tourangeau et al., 2002). In recent years, studies examining various health outcomes have benefited from IVR’s 24-h participant accessibility, reduction in staff burden, and data organization (e.g., Cash-Gibson et al., 2012). However, time is needed up-front in order to set up clear and succinct vocal prompts and validate scripts. In the CSDP, for instance, the first year of follow-up involved extensive staff time in telephone messaging to conduct individual 10-min telephone interviews. In the second year, IVR was introduced, with participant feedback indicating a preference for this method over telephone calls (section 3.2.4), and as technology rapidly evolved, IVR was then changed to secured internet surveys (see next section).

2.2.2. Web-based surveys

Higher follow-up rates than were previously feasible with traditional methods are now possible with web-based methodologies. In fact, recent data indicates that 84% of adults (ages 18 and older) and 74% of persons from lower socio-economic classes report internet use (Perrin and Duggan, 2015). For the participant, web-based surveys can be accessed anywhere at any time, and the inclusion of design elements, such as skip logic, can reduce time constraints and confusion. Research has also shown that participants are more likely to report sensitive behaviors with web-based surveys versus other modalities (Burkill et al., 2016). For the researcher, web-based modalities provide easy and cost-effective administration as well as rapid download for data analysis. Established survey services both publically available (e.g., Survey Monkey, Google Forms, etc.) and institutionally licensed (e.g., REDCap, Qualtrics, etc.) provide question response templates and confidential, encrypted data storage, which minimizes the initial setup time and cost associated with the need for outside programming assistance, knowledge of web applications, and required server access. Most surveys can be fairly easily adapted to an online format, however researchers now need to consider matching the original survey format with rapidly evolving changes in hardware preferences (i.e., desktop, tablets, phones, etc.; Toepoel and Lugtig, 2015).

Across all of our studies, online surveys now serve as the first-line method for follow-up and are administered as assessments ranging from brief substance use updates (5–10 min to complete) to multi-modular formats (25–30 min to complete). These may be complimented with subsequent face-to-face meetings for obtaining objective and biological measures. For example, in CSTOP, participants’ smoking status was first ascertained by online survey (or telephone, if preferred) and then followed by in-person biochemical confirmation for smoking abstinent participants.

2.2.3. Participant communication

While primers on study retention often emphasize the importance of regular participant communications (BootsMiller et al., 1998; Hunt and White, 1998; Twitchell et al., 1992), the details and customization of these communications are often not described. In our two larger studies, (CSDP and CSTOP), ongoing contact was maintained with participants by mailing quarterly study newsletters and annual birthday and holiday cards. Personal touches were included to encourage a sense of a study community (Lau and Lee, 1999) within these correspondences. For example, we created friendly staff photos that were relatively inexpensive to produce, by use of a digital camera, to include in secular holiday cards that incorporated winter images, popular cartoon characters, or Chicago sports themes. In newsletters, we regularly included interviews with staff about their backgrounds, hobbies, and future goals to foster a personal connection. Notably, these mailed items also served in participant tracking by including US postal service notice of “change service requested” on the envelope to obtain a forwarding address if a participant had moved. In addition, we included occasional puzzle contests with small monetary prizes to increase the likelihood of participants reading the newsletter. To respect participants’ time and attention, mere “check ins” for status updates were never conducted and communications were specific to newsletters, birthday/holiday cards, and/or notifications of an impending follow-up.

2.2.4. Incorporating smartphones in follow-up

One of the most transformative technological advances affecting everyday life has been the rapid adoption of cellular and smartphone technology. Nearly two-thirds of Americans and Western Europeans own smart phones with prevalent use across income and educational levels, age, and racial/ethnic groups (eMarketer, 2014; Smith, 2015). Hardware and software on these devices has also improved tremendously in the past decade to allow for tasks that would previously require a desktop computer. In our current era of social media and brief communications, we included concise messages to participants using cellular and smartphone technology for study reminders, or for providing links to study information or web-based surveys. For harder to reach subjects not responding to standard email or text notifications, we utilized the private, direct messaging options included in many social media platforms (e.g., Facebook™, Instagram™, etc.) with the knowledge that these communications might reach subjects attentive to notification prompts appearing on their phones (see next section).

2.2.5. Privacy concerns

While advances in technology have extended opportunities for data collection and participant communication, they have also raised privacy issues such as identity verification and determination of public versus private space. For example, participants who initially consented to providing contact information for future communications in 2004 could not have predicted the emergence of social media platforms (e.g., Facebook™) as publically accessible knowledge. Institutional review boards (IRBs) have reacted to privacy changes by developing more conservative policies about what qualifies as a violation of privacy and confidentiality, sometimes resulting in the need to amend protocol documents to account for these changes. In all of our work, in the midst of these evolving privacy concerns, secured data collection for participant confidentiality has been the top priority. For social media, while the studies maintained several social media accounts, only the nonpublic features (i.e., direct messaging) were utilized in communication attempts to avoid association with the study by publically visible posts or comments. In addition, our use of these sites always involved privacy settings to ensure that those connected were blocked from seeing other friends to protect identity. As technology continues to develop, it is imperative that research teams continually evaluate practical and theoretical implications of modality choices in order to foster trust with participants.

2.3. Mitigating non-responsive and hard-to-reach participants

As discussed in prior reviews (BootsMiller et al., 1998; Desmond et al., 1995; Twitchell et al., 1992; Wutzke et al., 2000), longitudinal follow-up inevitably includes non-responsive and hard-to-reach participants. In our studies, approximately 10% of participants have been particularly non-responsive and hard-to-reach. Although there is no a priori definition of a non-responsive participant (Cottler et al., 1996; Kleschinsky et al., 2009), such participants are usually passively non-responsive to repeated contact attempts and do not complete their follow-up on time. The follow-up timeframe depends on the specific time-sensitive nature for each study. For example, in our own work, the follow-up interval which ranged from > 1 day past due in CYAHS, > 1 month in CSTOP and EASS, and > 2 months in CSDP. The pacing of contact attempts usually increased 2–3 folds in such instances.

Research has also shown that non-responsive participants tend to have more psychiatric comorbidities or substance abuse issues than responsive participants (de Graaf et al., 2000; McCabe and West, 2016). Unfortunately, it is often difficult to forecast at the outset of longitudinal research which participants will be difficult to track over time and ruling out such persons might compromise the integrity of recruitment (Kipke et al., 1997; Plane and Jurjevich, 2009; Solorio et al., 2008). Thus, modification of a follow-up protocol for challenging participants can provide necessary flexibility (Kilsdonk et al., 2015). These modifications can include a waiting period before reinitiating contact attempts or having participants elect a one-time “opt out” of a follow-up wave, as we used in our CSDP study, but have only needed to apply in < 0.004% of all follow-ups thus far.

Given such non-responsive participants, how did we achieve 94–99% follow-up (see section 3) across our studies? First, we utilized the retention methods described in this review, with increased communication attempts for non-responsive participants and contacts to collateral persons, as warranted (see section 3.2.5). Second, our team brainstormed how to incorporate broader follow-up methodology in order to reach passive non-responders beyond the standard modalities (Table 3) to include social media, online publicly accessible information, or home visits. Third, we adopted a positive and understanding demeanor while responding rapidly to the needs of challenging participants (e.g., coordinating travel arrangements, child or pet care compensation, etc.). From the outset, we incorporated a motto of “no subject left behind” which helped to encourage project staff to maintain persistence and patience with challenging participants while documenting contact attempts (Desmond et al., 1995). When a participant was deemed particularly non-responsive, the PI would become involved and take primary responsibility for contact. This has been effective in the vast majority of cases and may be due to participants’ familiarity with the PI via a regular PI column and photo in newsletters, as well as selective response to an authority figure as has been noted in other research (Berkowitz and Lundy, 1957).

Table 3.

Examples of Online Resources to Locate Participants.

Typea	URL Examples	Purpose
Directories
White Pages	http://www.whitepages.com/	Address and contact information based on last known address, academic or work affiliation
Alumni Associations	http://alumniandfriends.uchicago.edu/ b
Organization/Work	http://directory.uchicago.edu/ b
Internet Phone
Google Voice	http://www.google.com/googlevoice/about.html	Free domestic and international calls and texting; virtual ‘face-to-face’ interviews
Skype	http://www.skype.com/en/
Facetime	http://www.apple.com/ios/facetime/
Maps
Google Maps	http://www.google.com/maps	Address confirmation, satellite views to facilitate home visits
Social Media
Facebook	/	Subject contact via site based messaging, current location, job and collateral information
Twitter	http://twitter.com/
LinkedIn	http://www.linkedin.com/
Instagram	http://www.instagram.com/
Mail Services
FedEx	http://www.fedex.com/	Current Address, authorized signing for location verification, “Return Address Service” for new location
US Postal Service	http://www.usps.com/
Miscellaneous
Dating Websites	http://www.okcupid.com/b	Current status, location, job, name changes, interests and hobbies, incarceration information, etc.
Wedding Registries	https://www.theknot.com/registry/couplesearchb
Offender Search	http://www.illinois.gov/IDOC/OFFENDER/b
Death Records	http://search.ancestry.com/search

^aReferences to information found in the table include: Bootsmiller et al., 1998; Cottler et al., 1996; Hunt and White, 1998; Kleschinsky et al., 2009; Scott, 2004; Twitchell et al., 1992.

^bExamples provided vary by institution, organization, and location.

3. Results

3.1. Follow-up rates

The procedures for mindset, modalities, and mitigation described in this review produced outstanding follow-up across our four studies in 697 participants. Despite the fact that the goals, design, and sample characteristics varied across these studies (Table 1), the methodology resulted in uniformly high follow-up rates. In the CSDP (King et al., 2011; King et al., 2014; King, 2016), the most extensive of our studies in terms of follow-up length, heavy and light drinker participants completed eight follow-ups in the first 2 years after their initial alcohol challenge laboratory sessions, and we achieved a 99% (1502/1520) follow-up rate for that interval. Through 6 years of follow-up, the overall 99% follow-up rate remained (i.e., 98% among heavy drinkers only) and continues at that rate through our 10-year follow-up (King, 2016). In our other laboratory study in emerging adult smokers, EASS (Conrad et al., 2013), at the one-year follow-up, we had a 95% follow-up rate (53/56). Likewise, in our clinical trials with sample sizes of 315 smokers (CSTOP; King et al., 2012) we demonstrated 99% rates of follow-up at 6-months and 97% rates of follow-up at 12-months and in our sample of 30 young drinker-smokers (CYAHS; Fridberg et al., 2014), we showed 97% and 94% follow-up rates at 1- and 6-months, respectively.

3.2. Effect of increasing incentives for on-time follow-up

Incentivizing “on-time” follow-up completion was employed in the CSDP to reduce staff burden and time. Completing a quarterly follow-up on the actual due date (the first day of the month) allowed participants to “double their rewards” with compensation of $20.00 as opposed to the usual $10.00. Results in the first three months of implementing this procedure showed that the first day of the month follow-up completion rates increased to 68% compared with 18% before the procedure was launched.

3.3. Follow-up rates when ideal pre-planning not possible

While an a priori follow-up framework is ideal, it is not always possible within our uncertain funding climate, grant cycles, and new discovery during a trial. These sometimes render researchers unable to put forth exact follow-up procedures for participants at the study outset. For example, ideal pre-planning was not available for our brief-intervention study (CYAHS). During enrollment, we informed participants of a planned one-month follow-up. However, as the study progressed, we extended data collection to also include a six-month follow-up. Upon receiving regulatory approval for this addition, we sent participants a letter and e-mail describing this addition and included extra reminders one week and one day prior to the follow-up. An additional e-mail was sent the morning of the follow-up to alert participants the survey link was active. Those who had not responded by mid-afternoon received further text, telephone, and email messages. With this more intensive protocol focused on notifications, we achieved 90% follow-ups completion on the exact due date, with the remaining 4% completing follow-up within two weeks. In addition, the follow-up rates at 1 month (97%) and 6 months (94%) did not differ.

3.4. Updating technologies for follow-up

In our two studies with samples of emerging adults, EASS and CYAH, over 95% of data was collected via online assessments with only 5% of data obtained by more traditional methods such as mail or telephone. Related, in CSDP, prior to implementing IVR procedures, follow-up involved staff time for repeated notification calls and ~10 min to conduct each interview once the date and time were scheduled. Following implementation of IVR in 2005, 76% (773/1015) of surveys were conducted via this method at the participants’ convenience, with the average IVR follow-up taking only 7 min to complete, an approximate 30% reduction in time compared with the telephone interviews. At the next juncture, after employing web-based follow-up, we conducted a feedback survey to assess participants’ preference for follow-up modalities. Results confirmed that participants felt that web-based surveys were more convenient than IVR (4.7 ± 0.7 vs. 2.8 ± 1.4 on a 5 point Likert scale, respectively). Thereafter, we transitioned fully to the web-based online format and IVR was discontinued.

3.5. Reducing drop-out rates in difficult-to-reach participants

Intensive measures and repeated contacts are often needed to achieve follow-up for a participant who is difficult to reach or initially non-responsive. In CSDP, data from the second heavy drinker cohort showed that 86% of the 104 participants completed their five-year follow-up assessment within the month it was due, with an average of 10.4 days from first notification to completion requiring 2.4 ± 0.8 SD notifications. For the 5% of the most challenging non-responsive participants, an average of 47 contact attempts (range 12–99) were needed to achieve successful completion of follow-up. After the tenth contact attempt, we routinely gave participants the opportunity to withdraw from the study, but we reminded them that this meant they would not be eligible for future study opportunities or compensations. Through 6 years of follow-up in the study, only 3/290 (1%) of participants have elected to drop out. Thus, persisting with challenging participants by encouraging the importance of their continued involvement resulted in a very high response rate and infrequent study withdrawal.

4. Conclusion

Achieving excellent retention across study types and sub-populations remains imperative to the scientific rigor of longitudinal research and is increasingly feasible in the digital age. Though there is no “one size fits all” approach, many of the methods we outline are not cost-prohibitive and yet highly portable across investigation types and clinical samples. While marginalized subgroups, e.g., homeless, transient, or severely mentally ill addiction subpopulations may present unique challenges for researchers, the principles outlined in this paper are portable and can be customized to the needs of such diverse samples. Of note, two of our four study samples were nearly half minority, and non-responsive and hard to reach participants were of the same racial breakdown as the larger study. This suggests applicability of our mindset, modalities, and mitigation methods across racial and ethnic subgroups. One limitation of our work is that the samples primarily consisted of individuals with at least with a high school education, so lower educated samples may need additional methods and support for follow-up. However, education level was not associated with follow-up in our work or that of others (e.g., Cottler et al., 1996; Gilmore, 2012).

In sum, this paper adds to the existing literature on retention strategies with a more detailed description of currently used methods and mindset, especially in the context of rapidly evolving technology and the need for researchers to keep pace with their widespread use. Our conceptual framework of mindset, modalities, and mitigating (difficult to reach participants) can be adapted in most, if not all, research settings. While quantifying the aspects of the study mindset were not possible per se for past research, we believe that it is the most crucial element in the groundwork for exceptional retention and a positive mindset was employed regularly in our mitigation of difficult participants. Our overall experience across four studies suggests that in order to provide the greatest accuracy in long-term data, it is imperative that these three elements are incorporated into the longitudinal research framework and adapted to enhance addition-based research methodologies in the 21st century.