Recruitment and Retention of Homeless Youth in a Substance Use and HIV-risk Reduction Program

Abstract

Conducting intervention studies with homeless populations can be difficult, particularly in terms of retaining participants across multiple sessions and locating them for subsequent follow-up assessments. Homeless youth are even more challenging to engage due to substance use, mental health problems, wariness of authority figures, and frequent relocations. This article describes methods used to successfully recruit a sample of 200 homeless youth from two drop-in centers in Los Angeles, engage them in a four-session substance use and sexual risk reduction program (79% of youth attended multiple sessions), and retain 91% of the full sample at a three-month follow-up assessment. Our experience indicates that utilizing structured project materials and having a small dedicated staff are essential to recruitment and retention efforts for intervention studies with homeless youth. Using these and other nontraditional methods are likely necessary to engage this at-risk yet hard-to-reach population.

Introduction

Low participation and retention rates in program evaluation research can threaten the internal validity of collected data, reduce power necessary for preferred analytic approaches, and preclude researchers from making strong claims about effects of the program. As such, recruitment methods have traditionally relied on advertisements (e.g., flyers, newspaper, or television ads), referrals (e.g., from providers or support staff), and participant-driven recruitment (e.g., respondent-driven sampling, snowball sampling), whereas retention methods in longitudinal studies typically include the use of incentives (e.g., cash, gift cards) to encourage participants to return for follow-up assessments (Becker et al. 2014; Hall et al. 2003; McKenzie et al. 1999). Achieving high participation and retention rates required of rigorously conducted evaluations can be particularly challenging when working with mobile and hard-to-reach populations. For example, recruitment and retention in longitudinal studies of individuals who are homeless (Glasser et al. 2014; Koegel et al. 1996) is challenging due to fluctuations in temporary residences (e.g., sleeping on a street one night, a shelter the next night, and a friend’s house the next night), presence of substance use and mental health problems, lack of permanent or mailing address, wariness of authority figures (which may include research field staff), or simply disinterest in the research.

A recent review of interventions targeted toward helping homeless youth engage in services and/or reduce engagement in risky behaviors found that many of these intervention studies reported suboptimal retention rates (Altena et al. 2010). For example, one study of an HIV-risk reduction program lost 35% of participants at six-month follow-up (Nyamathi, Branson, et al. 2012), and another study lost 24% at six-month follow-up, with participants completing an average of 7.2 treatment sessions out of an offered 16-session substance use program (Slesnick et al. 2008). Even when recruitment rates meet or exceed traditionally acceptable rates (e.g., 80%), a self-selection bias may be introduced if those participants who are most vulnerable or most in need of services are less likely to be retained in the study. For example, one study retained 80% of the sample at a three-month assessment point following a brief motivational intervention for substance use; however, significant differences were found between those who completed follow-up and those who did not, including important factors such as length of time on the street homeless, gender, time period when participants were recruited, and alcohol use, the latter of which was a targeted outcome of the intervention (Peterson et al. 2006).

To achieve adequate response rates, especially in a highly mobile and transient population, such as homeless youth (Ferguson et al. 2014; Martino et al. 2011), researchers must be creative and open to adapting traditional social science methods. Models exist to locate and track hard-to-reach populations in longitudinal studies, such as illicit drug users (Cottler et al. 1996; Scott 2004) and those with severe mental illnesses (Coen et al. 1996; McKenzie et al. 1999). However, few successful models exist to track homeless youth, and the existing models are over 15 years old (McKenzie et al. 1999). In addition, it is essential to better understand how to locate and track homeless youth involved in intervention studies, given that this population is in great need of effective programs to target problem behaviors such as alcohol misuse, illicit drug use, and risky sexual practices (Al-Tayyib et al. 2014; Nyamathi, Hudson, et al. 2012; Tucker et al. 2012; Wenzel et al. 2010).

The Present Study

This study adds to the literature on homeless youth field research methodology by describing the successful recruitment and retention strategies used for a randomized controlled trial of AWARE, a voluntary four-session group-based motivational interviewing intervention to reduce substance use and risky sexual behaviors among English-speaking unaccompanied homeless young adults between the ages of 18 and 25 (Tucker et al. 2017). To evaluate the efficacy of the AWARE program, we recruited 200 homeless young adults (100 in an intervention group, 100 in a control group), who were present during recruitment hours at one of the two participating drop-in centers in locations near Venice and Hollywood in Los Angeles. The study procedures involved youth completing a baseline survey, participating in the four-session AWARE program (if assigned to the intervention group), and completing another survey at a three-month follow-up. Given the difficulties inherent to engaging these youth in services, we needed to think creatively and develop a series of methods that would allow us to enroll youth in the program, achieve adequate intervention attendance, and obtain sufficient follow-up rates. We describe these methods in detail in the sections below.

Strategies for Recruiting Homeless Youth

Methods for Recruitment Strategies

Eligibility criteria included: (1) being between ages 18 and 25 years; and (2) seeking services at one of the participating drop-in centers. The two drop-in centers represent diversity in terms of location (i.e., urban/downtown vs. beach) and clientele (Golinelli et al. 2015). Due to the mobility of the homeless youth population, the study eligibility criteria additionally included the following in order to maximize our chances of locating the youth for follow-up: (3) willing to provide contact information; (4) having an e-mail account that they check regularly or a cell phone where they can receive calls; and (5) planning to be in the study area (i.e., Los Angeles County) for the next month, which would give them the opportunity to complete the four-session program if randomized to AWARE.

Recruitment occurred in four 16-week cycles, with the two drop-in centers alternating across cycles in serving as the intervention site or control site. Once per week for the duration of the project, a field supervisor and one or two staff members screened participants at the drop-in center. Participants in both conditions were recruited by advertising the study at the drop-in centers and through soliciting volunteers during recruitment visits. Youth signed up on a sign-in sheet each day to participate in the study. Depending on the number of youth who signed up, they were randomly selected from the pool of interested youth and screened for eligibility. Eligible participants provided written consent and completed a baseline self-report survey, which took approximately 30 minutes and was completed in the presence of a staff member if questions arose. Participants received US$20 for completing this baseline survey. All study materials and procedures were approved by the institution’s internal review board.

Results of Recruitment Strategies

We recruited youth in the intervention condition by conducting multiple visits to the drop-in center during the first half of each 16-week cycle. We started by recruiting five youth for the first group and then slowly recruited new youth for each of the first eight weeks until we reached our recruitment goals. This allowed us the flexibility of being able to add new youth to make sure we could always hold a group and still gave each youth the opportunity to attend to complete the four individual sessions over an eight-week period. For control group participants, the first two cycles required 21 youth each, and we reached our enrollment target after two visits to the drop-in center. The second two cycles required 58 youth each, and we reached our enrollment target in three visits at the drop-in center. Two youth screened out as ineligible, and one youth refused to participate at one of the drop-in centers (simply stating they “were not interested”).

Strategies for Retaining Youth in AWARE Program

Methods for Retention Strategies

AWARE was designed to be delivered in a small group setting. To manage the size of the intervention groups and the number of people who would be able to complete all four sessions, we conducted a rolling enrollment over the course of the first eight weeks of the cycle, guaranteeing that each respondent would have at least two opportunities to attend each of the four individual sessions. Participants could attend any missed session at any point during any cycle. To ensure that intervention participants presenting for a group were able to complete their session, if only one person was found to attend a session, a second respondent (in many cases, someone who had either already “graduated” or who had attended that particular session) was approached and invited to participate to avoid holding a session with just one participant. This only occurred 14 times over the course of the study. If we could not locate anyone to do the scheduled session, we had the flexibility to change which session we delivered to maximize the number of youth who could attend and increase their chances of graduating. For example, if two participants who received sessions 1, 2, and 3 were present, and we were scheduled to deliver session 2 that day but no eligible participants for session 2 were available at the drop-in center, we would deliver session 4 to graduate the two available participants interested in receiving session 4.

Participants were given a US$5 incentive for each session attended and a US$15 bonus for attending all four of them. We also offered light refreshments and free condoms. We experimented with offering a pizza party on a monthly basis for all graduates who completed all four sessions but found that it was hard to get all of them to come on that specific day. The youth preferred the US$15 bonus for attending all four sessions instead.

Results of Retention Strategies

Each of the four AWARE sessions was delivered at least twice, and some sessions were delivered as many as five times during the 16-week cycle. After the 11th week of the intervention cycles, some sessions were canceled because no eligible participants were located to attend. This happened five times in the urban site (16% of scheduled sessions) and twice in the beach site (6% of scheduled sessions). Over the course of the study, the intervention group sizes ranged from two to 10 youth in the urban site, with an average group size of four, and from two to 13 youth in beach site, with an average group size of five. Ultimately, 21% of participants attended one session, 27% attended two sessions, 4% attended three sessions, and 48% attended all four sessions and graduated the program.

Strategies for Tracking and Locating Participants: Adapting Standard Methods

Methods for Tracking and Locating Participants

Due to the specific nature of this population, standard tracking and locating techniques had to be adapted to maximize the chances of locating and completing the follow-up survey. In Table 1, we detail our strategy for tracking and locating participants compared to the standard practices for following participants in longitudinal studies (Becker et al. 2014; Hall et al. 2003; McKenzie et al. 1999). We list how we adapted strategies around telephone attempts, mailings, field visits, use of public records, use of social networking sites, and use of service providers. We also developed a series of tools to assist with tracking over time in both the intervention and control conditions. We did not collect data on which specific strategies and tools were most successful because we used a combination of multiple methods in each case. Further research is needed to examine more explicitly how each of these methods can be used to reach this population either alone or in combination and to examine the demographics/characteristics of participants found via each method.

Table 1.

Description of Standard Practices and Strategies Used in the Present Study to Track and Locate Homeless Youth Participants for Follow-up.

Tracking and Location Method	Standard Practice	Strategy Used in the Present Study
Telephone attempts	Calling at varied times of the day and days of the week	We had more success calling during targeted set times, usually between 11 a.m. and 4 p.m. These hours correspond generally with the hours of the drop-in center, when youth had the opportunity to charge their phones Due to the inconsistent nature of this population’s sleeping locations, early mornings were not productive times to call Due to the alcohol and substance use of this population, by evening, many were too intoxicated to complete a survey Many times, cell phones would get disconnected and then reconnected early in the month, so the end of the month was not a good time to call
Mailings	Mailing letters and postcards to addresses given at baseline	None of the youth in the sample had a reliable mailing address. Mailings were only useful to third-party contacts or service center contacts We could mail to a service center that collected mail for homeless youth, but this was not productive as most youth did not respond to the mailings and we could not even be sure that they received them
Field visits	Visiting addresses at varied times during the day	There were no addresses to visit besides drop-in centers and known hangouts We had success doing random visits to each of the drop-in centers The field staff, wearing the AWARE T-shirts, would walk the streets in the two study areas looking for participants or friends of the participants Often we collected tracking information about hangouts and places used for sleeping outdoors; field staff would visit these locations regularly
Use of public records	Internet searches on free sites such as Google.com or Spokeo.com as well as paid services such as LexisNexis or Experian	Typical public records were not productive for locating the youth in the sample These resources work better for housed, more established populations (e.g., those who apply for credit) In many cases, these sources are valuable in locating family members and other thirdparty contacts; however, they proved to be not useful for this population and most third-party contacts had not recently seen or heard from the youth The Los Angeles County Jail website https://app4.lasd.org/iic/ajis_search.cfm was a very useful resource as some of the youth were arrested during the field period
Use of social networking sites (SNSs)	There is not an established set of standard practices in using SNSs	Facebook could be an effective way to locate and make contact with these participants Many homeless youth access the Internet at libraries and at the drop-in centers and use Facebook to communicate with friends and family since it is a free service (as opposed to paying for a cell phone plan) We asked for SNS usernames on the initial tracking and locating form and obtained permission to contact them via Facebook or Twitter We monitored their profiles to see if they posted public information about their location, travel plans, and in some cases posted e-mail addresses or phone numbers We only sent personal (nonpublic) Facebook messages to participants; this costs US$1 per message to get it delivered to their regular notifications and not their spam mailbox We did not “friend” youth on Facebook due to internal review board concerns
Use of service providers	Build relationships with staff at local service providers	Visiting the drop-in centers and relying on information gained from volunteers and agency staff was invaluable Visits to the drop-in during the first hour of opening and during meal times (i.e., most popular times) were very productive Many participants were consistent users of the drop-in services Agency staff knew a lot about the youth’s friends, habits, and recent activities We distributed “difficult-to-find” lists and were often contacted in real time when a participant arrived at the center

Incentives.

First, participants could complete the three-month follow-up survey either in-person or over the phone assisted by one of our field staff (e.g., for those youth who had moved out of the study area), for which they received a US$30 cash payment. Youth who completed follow-up surveys over the telephone were mailed a postal money order or, in one case, wired money via Western Union. Participants who completed the survey from the Los Angeles County Jail had the US$30 payment put into their jail accounts or “books.”

Tracking and locating form.

We collected traditional information such as e-mail addresses, telephone numbers, and mailing addresses for participants and two friends/relatives who knew how to reach them. We also obtained permission to text their cell phone numbers. This proved to be a valuable way to make contact because many youth would not answer their phone if they did not recognize the number calling. We also collected other information that would be useful for a population that by definition had no permanent home address. Some of these features included getting their “street” name or nickname, which is often how their peers and the service provider staff knew them best. We also asked if they had a Facebook or other social networking site account and, if so, obtained the name associated with that account and permission to message them. We recorded physical descriptions of each participant including gender, ethnicity, height, hair color, eye color, and permanent identifying characteristics (e.g., tattoos, scars). We obtained detailed descriptions of their hangouts, places they went to eat, and places they regularly went for services (e.g., mental health clinics or shelters). We recorded the names of caseworkers, staff members, or volunteers that they saw regularly at the drop-in center where they were recruited. Participants indicated what the best way would be to contact them for the follow-up survey, including ways that we had not specifically asked them about.

Project reminder card.

Each participant was given a branded AWARE project reminder card and told to think of it as a “US$30 bill” that they should put in their wallets to be redeemed in three months. These cards were printed in color with the project logo and toll-free project hotline number printed on it. The project logo and toll-free phone number were also displayed on posters and flyers posted at each of the drop-in centers. Additional cards were given at each contact, such as during a scheduled interim contact, after attending an intervention session, or when seen on the street or at the drop-in center. The supervisor and field staff carried a hard copy list of participants’ names and target dates, so the date of their follow-up survey could be written on the card at each contact. We also added an additional US$5 incentive if the participant called or contacted us prior to their scheduled date to schedule the follow-up survey. During the course of the field period, a number of participants stopped us on the street and showed us they still had their card (we did not track how many did this), but only three participants actually used the number or e-mail to contact us prior to their scheduled follow-up date and received the extra US$5 incentive.

Difficult-to-find list.

We identified participants who were the most difficult to locate and placed them on a list that all field staff carried. The list included all youth who had passed their three-month target date for follow-up completion. Every time we made contact with a drop-in staff member, we would ask when the last time each person on this list was seen and left notes for them to contact us. There were about 15 cases during the course of the field period where field staff ran into a member of the list and were able to complete the follow-up data collection. Sometimes, this occurred during a visit to one of the drop-in centers, and other times it happened while visiting known “hot spots” in the community, such as popular fast-food restaurants or meal lines. In addition, all participants on the difficult-to-find list were searched on the Los Angeles County Jail website daily.

Third-party contacts.

For this population, third-party contacts were only valuable in specific situations. About half of the youth gave us the contact information of friends or family. Because they were currently homeless, their relationships with family members from home were usually strained, and contacting those family members yielded little in terms of current locating information. There were between 15 and 20 situations, however, where we discovered from friends or drop-in center staff that a youth had gone back home, and we were able to reach a family member and complete the survey over the telephone. The other third-party contacts given that were most helpful included other homeless youth members of their “traveler families” and, occasionally, counselors from one of the local service centers with whom they stayed in contact. In all cases, we did not mention the homeless status of the participant when communicating with third-party contacts, merely relaying that we were looking for them to complete a survey for “a paid health study that they were participating in” and that the participant gave us their name and contact information as a means of locating them.

Field staff flexibility.

All field staff wore T-shirts with the project logo at every visit. The project staff were extremely knowledgeable about the local homeless hangouts including libraries, coffee shops, and fast-food restaurants frequented by homeless youth, and all had extensive experience working on the streets in Hollywood and on the Venice boardwalk (i.e., hot spots for homeless youth in Los Angeles). Their success in establishing rapport with participants (e.g., gaining their trust, not being judgmental) resulted in 62 surveys being completed on the streets or beach where the field staff would walk the streets or beaches and see a participant, and the pair would move to a private area to complete the survey. In some cases, the field staff would stop whatever other work they were doing to meet a participant and complete a survey. For example, in at least 10 cases, volunteers or staff members would contact us when someone we had been looking for arrived at the drop-in center and there were approximately seven times when a participant called to say they were at a specific place and wanted to do the survey immediately so they could get the incentive. The field staff would ask where they were, tell them to stay there, and immediately go meet them. In one memorable case, a staff member received a call from a participant who said she was at the freight train depot in Ventura about to hop a train back to Utah where she was going to have her baby. The staff member immediately got in the car and drove the hour north to complete the survey and finished 20 minutes before the train departed. In nine other cases where a participant was located on the Los Angeles County Jail website, the staff member would rush to the jail to complete the survey before the participant was released, which was typically within 24–48 hours.

Team communication.

Team meetings were held twice weekly where the field team had an opportunity to share their real-time experiences with the research management staff. This included production updates and the designation of problem cases to be added to the difficult-to-find list. The meetings also provided a forum to discuss successful tracking efforts that could be shared with other field staff, an opportunity to transfer field cases among each staff member when it became apparent that the participant was avoiding the initially assigned staff member and for the project coordinator to give feedback and offer tracking advice on a case-by-case basis. Meetings were also an opportunity to report on what was going on within the homeless community that may be affecting the areas, where homeless youth would congregate or sleep or impact use of the drop-in centers. For example, during the course of the project, there were several shootings of homeless youth by the Los Angeles Police, and the areas where we were conducting the research were often blanketed by police officer and business improvement district officer activity. Knowledge of these incidents provided context to field staff about why some weeks we saw few attendees at AWARE groups as well as helped field staff during site visits be sensitive to youth affected by these incidents.

Results of Strategies for Tracking and Locating Participants

Using these various methods, we were able to achieve an overall response rate of 91% at the three-month follow-up. We located two youth who could not be surveyed because they were incarcerated out of state and an additional three youth who refused the follow-up survey. One of the refusals was located at his new job and did not want complete the survey because he said that he no longer needed the money. The two others were a couple who responded to a message on Facebook saying that they had left the state and did not want to continue participating in the project. In total, there were only 14 youth (7%) who we were unable to locate at the three-month follow-up. Of our 200 participants, we located 71 at the drop-in center where they were recruited (39%), 66 on the street or in the community (36%), nine in Los Angeles County Jail (5%), and 35 who had already left the area and were completed over the telephone (19% of 200 participants). Of those who completed the follow-up by telephone, surveys were completed in 12 different states and eight different California counties. Table 2 contains the location of where participants completed surveys by intervention and control conditions.

Table 2.

Location of Follow-up Survey Completes by Condition.

Site	Total	Drop-in Center	Jail	Telephone	Street	Transitional Housing
Intervention	95	37	8	13	34	3
Control	86	34	1	22	28	1
Total	181	71	9	35	62	4
Percentage (%)		39	5	19	34	2

Follow-up rates were variable between sites, as more youth at the urban site (94/100) completed the study than youth at the beach site (87/100) and more intervention youth (95/100) completed the study than control youth (86/100). Part of the challenge of the follow-up data collection effort was that for the control group, we only had one contact with them at baseline, whereas intervention participants had the opportunity to be reminded about the follow-up survey during every contact with them (e.g., at group sessions, during reminders to attend the group sessions). Compared to the control group, participants in the intervention group received an additional four-16 interim contact attempts. For this reason, it was very important to have a motivated field team who were able to build rapport from the initial interaction, convince the respondent to provide as much information as possible, clarify the incentive structure, and be creative in engaging the respondent about what the best way to find them would be.

Conclusion

Designing an intervention study targeted for at-risk homeless youth can seem daunting due to their high mobility, their ability to remain invisible to standard tracking strategies, and their tendency to avoid authority figures (such as researchers). Tracking the population of homeless youth is further complicated by the fact that many have drug and alcohol issues, are often involved in risky behaviors, and may have encounters with law enforcement. We discovered, however, that by developing a series of strategies and tools to reach and retain homeless youth, there are approaches that can be used with great success to retain a high percentage of the population. Using a combination of these strategies, we were able to encourage the sample of homeless youth to return for intervention sessions and complete a three-month follow-up assessment. We believe that these strategies can be replicated in future studies, and researchers are encouraged to use these developed strategies to conduct research with and provide further intervention and outreach to the population of homeless youth.