Retaining a Sample of Homeless Youth

Abstract

Objective

Hard to reach populations need to be included in research studies to ensure proper representation of the general population. This paper explores tracking strategies used in the Youth Matters in London project to retain a sample of homeless youth.

Method

A total of 187 youth, aged between 16 and 24 years, homeless or precariously housed, and experiencing a serious mental health issue were recruited at a community drop-in center, by word of mouth and by snowball sampling. After the initial interview, three repeat interviews were conducted six months apart.

Results

The most successful strategy for contacting participants was through a local agency and e-mail. An analysis of the contact data identified participant retention rates as 88%, 86%, and 82% for each successive interview. This longitudinal retention rate is very high compared with research in other vulnerable populations, suggesting a strong willingness to participate in the Youth Matters in London project.

Conclusions

Retaining a sample of homeless youth is difficult, however, with time, patience and effort it has proven possible. This research underscores the importance of relationships with community agencies to retain vulnerable youth samples in longitudinal research designs.

Across Canada, it is estimated that 40–65 thousand youth between the ages of 16–24 experience homelessness each year (Gaetz, O’Grady, Kidd, & Schwan, 2016; Evenson, 2009) making up 20% of the homeless population accessing shelters (Segaert, 2012). It is estimated that 50–70% of persons who are experiencing homelessness have mental health issues which may include substance use disorders (Drake, Osher, & Wallach, 1991). Padgett, Gulcur, and Tsemberis (2006) identify that numbers are considered to be much higher than this due to under-reporting. Homeless youth have been identified as a highly transient population (Ferguson, Jun, Bender, Thompson, & Pollio, 2010; Hyde, 2005). More than 65% self-identify as nomadic (Sanders, Lankenau, Jackson-Bloom, & Hathazi, 2008), making the tracking of research participants a very challenging task.

Frequent address and phone number changes (Ward & Henderson, 2003), and limited conventional social networks (Falci, Whitbeck, Hoyt, & Rose, 2011) make maintaining current locator information difficult (Farabee, Hawken, & Griffith, 2011). Nevertheless, it is important to include members of hard to reach populations in research to ensure proper representation of the general population and to explore their specific needs. Odierna and Schmidt (2009) found that participant retention rates were lowered from 89% to 71% twelve months after baseline when hard to reach populations were included. In research using both housed and homeless adolescents, participant retention rates were 92% and 80% respectively four and a half years after they were first contacted (Hobden, Forney, Durham, & Toro, 2011).

Retention Strategies

Locator information is generally the key to maintaining participants in a longitudinal study (Robinson, Dennison, Wayman, Pronovosta, & Needham, 2007). Researchers are most successful in tracking participants when all possible location information is obtained, such as full names, aliases, social insurance and driver’s license numbers, current or best mailing addresses, phone numbers, email, phone numbers and email of participant’s contacts, places frequented, involvement with social services or judicial systems, and any plans to move (U.S. Department of Health and Human Services, 2006). Collecting “collateral” contact information, i.e. contact information of those who might know the participant’s whereabouts, is a very successful locating strategy (Hobden et al., 2011). Updating locator forms at each interview (Cottler, Compton, Ben-Abdallah, Horne, & Claverie, 1996) and keeping in contact with participants between interviews (Robinson et al., 2007) are useful strategies to maintain current information. A 24 hour message line can be a useful way for participants to call and update their locator information, schedule interviews, or disclose other needs related to the study (McKenzie, Peterson, Tulsky, Long, Chesney, & Moss, 1999).

Email

Email is also an effective means of communication for street youth (Lankenau, Sanders, HatHazi, & Jackson Bloom, 2009). In a Canadian study of 20 street youth, approximately 70% of the sample self-assessed their computer abilities on a structured interview as “better than average” and 85% indicated comfort using computer technology (Karabanow & Naylor, 2010). This sample of youth reported “feeling connected” through the internet and access to computers was identified as primarily being available at drop-in centers or public libraries, where they could check their email. Further studies (Rice, Monro, Barman-Adhikari, & Young, 2010; Rice & Barman-Adhikari, 2014) found 80% of youth experiencing homelessness accessed internet more than once a week and as much as 25% accessed internet for one hour or more daily. The availability of public internet access through libraries, smart phone technology, and public wifi increases accessibility amongst this population. Rice, Lee, and Taitt (2011) found only 22% of homeless youth did not have access to a cell phone. In another study by Eyrich-Garg and Moss (2017) the use of the internet by homeless participants at public libraries, places of employment and on their mobile phones, improved communication between participants and researchers. These studies highlight the increasing accessibility of the internet and the importance of collecting email addresses when tracking homeless youth. Street youth are generally able to check their email through a variety of means, although less frequently than a housed population.

Social service agencies

Tracking participants through social service agencies can be difficult due to staff turnover, and confidentiality and consent issues when using agency information (Ward & Henderson, 2003). Agency tracking may be more successful when the agency is directly involved in the research process, and deriving benefit from this involvement (Pollio, Thompson, & North, 2000; Veldhuizen et al., 2015). These considerations motivate the agency’s tracking efforts in collaboration with the research team (Pollio, Thompson, & North, 2010; Veldhuizen et al., 2015). Creating and maintaining strong relationships with social service or health care agencies in the community can be beneficial in locating participants through formal agency databases or other informal means (McKenzie et al., 1999).

System tracking

Bureaucratic obstacles aside, system tracking, such as accessing judicial records or other public and private service records, can be very effective in tracking the whereabouts of participants. However, accessing such databases often proves difficult, again due to issues surrounding confidentiality and consent (Cottler et al, 1996; U.S. Department of Health and Human Services, 2006). Furthermore, these records tend to leave out certain populations in their reports (Williams & O’Donnell, 2014). For instance, criminal records databases may be the easiest type of database to access, but these databases leave out individuals who have not been convicted of any crimes (Williams & O’Donnell, 2014).

Outreach

When researchers are unsuccessful in locating participants through regular contact methods, fieldwork or outreach may be effective (Cepeda & Valdez, 2010; Cottler et al., 1996; Lankenau et al., 2009; McKenzie et al., 1999; Robinson et al, 2007; Villacorta, et al., 2007). Outreach entails researchers visiting known hangout spots, social service agencies, drop-in centers, shelters, and neighborhoods related to the study population, as well as hand delivering letters or reminders (McKenzie et al., 1999; Villacorta et al., 2007). Field visits also have the indirect benefit of researchers maintaining presence within the population, building rapport, and monitoring participants’ locations, even when they are not due for interviewing (Villacorta et al., 2007).

Outreach specialists, carefully selected and specifically trained to the needs of the population, have been successfully utilized in a number of studies (McKenzie et al., 1999). For example, in tracking research on Mexican-American heroin users, Cepeda and Valdez (2010) used culturally appropriate outreach specialists, native to the neighborhood of participants. The retention success of Villcorta et al.’s (2007) research, involving socially marginalized young Peruvian adults, was attributed to extensive outreach efforts with researchers developing a vast understanding of the participants’ lifestyles before beginning research. “Helper participants” were also employed in Villcorta et al.’s (2007) study to spread the word of upcoming interviews at local participant hangout spots, and additional visits were scheduled at four, eight, 16 and 20 month intervals, specifically to locate participants and update their locator information. Furthermore, their field visits allowed for monitoring of participants, while outreach workers also took part in local events to maintain relationships and locate participants. Outreach workers create a network among the participants and the researchers while also helping to locate individuals via the network (Lankenau et al., 2009).

Incentives

Providing incentives to participants has proven to be an effective retention strategy (Cepeda & Valdez, 2010; McKenzie et al., 1999; Robinson et al., 2007; Veldhuizen et al., 2015). Incentives can range in amounts and come in cash or gift certificate form. Studies using incentives have had higher retention rates than those without (Cepeda & Valdez, 2010; Lankenau et al., 2009; McKenzie et al., 1999; Robinson et al., 2007). Immediate cash incentives have proven to be most effective (Cepeda & Valdez, 2010), and increasing the amount with each follow up interview is an additional useful strategy (Lankenau et al., 2009; Veldhuizen et al., 2015). Other incentives such as free counseling services or personalizing interviews to specific participants have also demonstrated increased retention rates (McKenzie et al., 1999). However, these incentives strategies may be perceived as coercive and an infringement of the right to voluntary consent.

Organizing tracking information

Documentation of both successful and unsuccessful contact attempts are effective in tracking exactly when and where participants can be found. Researchers have used spread sheets to organize this information so that participants’ past and current locations can be easily referred to (Cepeda & Valdez, 2010; McKenzie et al., 1999). Time frames for re-contacting vary among studies, but it has been useful to begin re-contacting participants at least a month in advance, in an effort to give enough time to locate individuals and also determine a day and time for their interview (Ward & Henderson, 2003; Yeterian, Dowa, & Kelly, 2012).

Study logo items

Sending various items to participants, such as thank you cards, reminder cards, newsletters, calendars, and birthday cards with gift certificates are effective retention strategies with a personal touch (Cepeda & Valdez, 2010; McKenzie et al., 1999; U.S. Department of Health and Human Services 2006; Yeterian et al., 2012). Giving participants study identification may increase retention as the participants value their involvement in the research. Using study logos has also proved effective; participants can recognize the logo in various locations and contact researchers directly (Villacorta et al., 2007; Walton, Ramanathan, & Reischi, 1998). Study logos can also be displayed on items that are given to participants in longitudinal studies, such as on pens and grocery bags, which can act as an incentive to participate (Holt et al., 2015).

Authentic relationships

Karabanow, Kidd, Frederick, McLuckie, and Quick (2016) addressed the question of longitudinal studies resulting in the development of authentic relationships between researchers and subjects by identifying the use of an anti-oppression framework in research. An anti-oppression framework is one used within disciplines including social work, anthropology, sociology, nursing and psychology. This framework allows the researcher to acknowledge and understand the power dynamics within the relationship as well as the privilege that is particular to the position of the researcher (Baines & Edward, 2015). Objective research techniques conventionally used do not allow researchers to recognize oppressive privilege within research methods, relationships and practices (Baines & Edwards, 2015). Cottler et al. (1996) achieved a retention rate of 96.6% in a vulnerable sample by using locator forms, phone tracking, and persistent and extensive outreach. Cottler et al. (1996) noted the importance of budgeting to allow for costly extensive outreach efforts. These researchers also used system tracking, meaning they were able to track their participants through various systems, such as the social welfare or criminal justice systems, which proved to be effective. Two “trackers” were employed for the duration of their study, allowing for consistent and involved outreach workers, which proved effective for their research.

Researchers cite persistence, creativity, and flexibility as basic but important aspects when tracking transient populations (Cottler et al., 1996; McKenzie et al., 1999). Outreach workers must exhaust all avenues in order to locate the most difficult to locate participants. Researchers must also be flexible in regards to scheduling and locations of interviews (Cottler et al., 1996). Ease and location of the interview is important to consider when working with homeless youth, as it could influence retention rates (Robinson et al., 2007). In the current study, tracking strategies including social service agency, email and outreach was utilized. Retention strategies involved documenting tracking information and giving out items with the study logo on it.

Methods

The research team did not set out at the beginning of the study to report tracking and retention strategies as a primary objective. The main objective of the Youth Matters in London, a participatory action research (PAR) project, was to investigate and better understand youth participants’ choices regarding mental health and addictions treatment and service options over a three-year period. The inclusion requirement for this project was that participants must have serious or moderate, self-reported mental disorder with or without a co-existing substance use disorder. It was not a requirement to have been formally diagnosed with a mental health or an addiction disorder. Participants were given an option to choose help with housing-first, treatment-first, both together, something else if preferred and a further option of changing their initial choice across the study time period. Refer to Forchuk et al. (2013) for the findings from this objective. The overall enrollment for the study was 187 youth from the City of London, Ontario, and surrounding areas. The age requirement for enrolling in the study was 16 to 24 years, self-reporting mental health issues and/or addiction issues, and currently homeless, or precariously housed. The definitions of “homelessness status” and “precariously housed” were consistent with the At Home/Chez Soi Project: “not having a place to stay for more than seven nights and having little chance of finding a place to stay in the next month” (Mental Health Commission of Canada, 2012).

Procedures

Ethics approval was granted by the Research Ethics Board of Western University before study commencement. This included a complete ethical review of the Letters of Information and Consent, which were explained to participants by research staff and signed to acknowledge their voluntary consent prior to conducting interviews. Recruitment was facilitated by a community partnership with the Youth Action Centre (YAC), a drop-in centre for homeless youth with street involvement operated by Youth Opportunities Unlimited (YOU). YAC dedicated one worker specifically to address housing and/or treatment needs of homeless youth involved in the study. Although this position was funded directly through the research grant, the incumbent was not a part of the research study team; no participant information was shared between the worker and the researchers. The ethics application referred to this additional worker to increase capacity for the project. Word of mouth and snowball sampling were also successful recruitment strategies. Although we planned to recruit a minimum of 150 youth in the recruitment phase, we recruited 187.

Analysis of contact approaches was done by referring to research coordinator field notes and contact lists (which modeled Cottler et al.’s (1996) locator forms) provided by participants at each interview. Contact information typically included personal phone number (land line and/or cell), e-mail address, and current “address” (often shelter or friends etc.). Contact information for family and/or friends included names, relationship, phone numbers, and email addresses. Agencies used by participants were also identified by organization name, phone number etc. and a contact person or case worker (when appropriate).

A combination of strategies was used to locate participants. Outreach approaches were used when contact information that had been provided failed to locate participants; this outreach modeled Cottler et al.’s (1996) approach. Outreach, sensitive and respectful to the needs of participants, consisted of designated research assistants visiting areas where homeless youth were known to gather to locate participants for interview. These sites included social service agencies, social assistance offices, and other popular meeting places. No concerns were expressed regarding surveillance. The use of promotional items, such as bags and lanyards with easily recognizable logos, were used to identify research assistants and participants. These lanyards were suggested by the youth on the advisory group and also given to youth at each interview. They included a phone number and internet site which could be used to access the study and in at least a few instances the lanyards were actually used by participants.

Results

Demographic information is presented in Table 1. The sample was made up of 122 (65%) males and 62 (33%) females, with a mean age of 20 years (SD = 2.4). On average, participants reported experiencing 24 months (SD = 28.27) of homelessness throughout their lifetime. Substance-use disorders (34.2%) and mood disorders (31.6%) were the top two psychiatric diagnoses reported by participants.

Table 1.

Homeless Youth Sample Characteristics

Variable	n (%)
Age (years) [Mean (SD)]	20.9 (2.4)
Gender
Male	122 (65.2)
Female	62 (33.2)
Other	3 (1.6)
Country of Birth
Canada	175 (93.6)
USA	4 (2.1)
Other	8 (4.3)
Level of Education
Completed grade school	149 (79.7)
Completed high school	27 (14.4)
Some post-secondary	11 (5.8)
Marital Status
Single, never married	146 (78.1)
Cohabitating with partner	37 (19.8)
Married	1 (0.5)
Separated	1 (0.5)
Don’t know	2 (1.1)
Current Primary Employment Status
Employed	13 (7.0)
Unemployed	140 (74.9)
Student	31 (16.6)
Volunteer work, unpaid	1 (0.5)
Other	1 (0.5)
Don’t know	1 (0.5)
Current Sources of Income
Welfare/Income assistance	95 (50.8)
Personal Needs Allowance	29 (15.5)
Other	17 (9.1)
Earnings from casual work	16 (8.6)
Pan-handling	13 (7.0)
No income	12 (6.4)
Partner/Family support	11 (5.9)
Disability income	9 (4.8)
Earnings from regular work	7 (3.7)
Illegal means	7 (3.7)
Busking	5 (2.7)
Ontario Student Assistance Program (OSAP)	3 (1.6)
HST/Income Tax rebate/cheque	3 (1.6)
Children’s Aid Society support	3 (1.6)
Baby bonus	2 (1.1)
Squeegeeing	2 (1.1)
Pension	1 (0.5)
Total income in the previous month (n=186) [Mean (SD)]	$970.55 ($2339.55)
Primary Diagnosis
Substance-related disorder	64 (34.2)
Mood disorder	59 (31.6)
Disorder of childhood/adolescence	25 (13.4)
Anxiety disorder	22 (11.8)
Mental health diagnosis present but type unknown	9 (4.8)
Schizophrenia	5 (2.7)
Other	2 (1.1)
Developmental handicap	1 (0.5)
Total amount of time spent being homeless during the lifetime (months) [Mean (SD)]	24.39 (28.27)
Longest single period of homelessness (months) [Mean (SD)]	12.22 (21.29)
Number of undesirable moves in the previous 5 years (n=179) [Mean (SD)]	9.09 (17.18)

Following initial enrolment, attempts were made to contact each participant for three follow up interviews, six months apart. Table 2 summarizes the overall retention rates for the study. Of the 23 participants lost to follow-up at the first follow-up interview, one person passed away, one person declined to participate in further interviews, and the remainder could not be contacted. The project team was therefore, successful in retaining 164 youth (88%) for the first follow-up (interview two). In subsequent interviews three and four, four and six participants (respectively) were lost to follow-up because they could not be contacted. Out of those that were contacted, all consented to participate in the interviews. The retention rates for interview three and four were 86% and 82% respectively. As illustrated in Table 2, the biggest drop in participant retention occurred between the first and second interview. In terms of participant retention between interviews, 97% of those participants who completed interview two also completed interview three, and 96% of those who completed interview three also completed interview four. These retention rates over time suggest a strong willingness by homeless youth to participate in the research study.

Table 2.

Retention Rates of the Homeless Youth Sample Following Successive Interviews

Interview No.	N	Lost to Follow-up	n	%
1 (baseline)	-	-	187	100
2 (6 months)	187	23	164	88
3 (12 months)	164	4	160	86
4 (18 months)	160	6	154	82

In total 665 interviews took place. This number includes the initial 187 at enrollment, and 164, 160, and 154 at each successive follow up interview. Table 3 summarizes the different contact strategies/methods which were successful in locating participants for each interview. As indicated by Table 3, the most successful method of contacting participants for all follow-up interviews was through the YAC, where 371 (78%) of the follow-up interviews were scheduled. Of the 371 interviews, 20 were contacted through the YAC outreach program (differentiated from YAC’s main operating location) (see Table 3). Emailing participants was the second (albeit a distant second) most successful strategy at relocating participants for follow-up interviews, as 26 (5.4%) of follow-up interviews were scheduled through this method, which is also shown in Table 3. Contact methods through phone (own) and outreach work done directly by the project research team were the least successful as 18 (3.8%) and 11 (2.3%) participants were scheduled for interviews three and four respectively, using these methods. For 41 (8.6%) of follow-up interviews, a single contact method could not be determined, as multiple methods were utilized. For example, one participant mentioned having received both a message from a sibling and an email, but then came to the YAC to find a member of the research staff. Despite multiple contact methods being used in some cases, the primary contact methods for over 90% cases were identified. The presence of successful contact methods other than the YAC emphasizes the need to keep ongoing detailed records at each point in time for both the documentation of successful strategies and research purposes.

Table 3.

Successful Contact Strategies for Retaining the Sample of Homeless Youth

Interview No.	Phone (own)	Phone (family or friend)	Email	YAC	Outreach	Jail / detention	Unknown	Total
Two	3	3	4	137 (5 Outreach)	1	1	15	164
Three	5	2	6	128 (8 Outreach)	3	2	14	160
Four	10	0	16	106 (7 Outreach)	7	3	12	154
Total	18	5	26	371 (20 Outreach)	11	6	41	478

Discussion

A major strength of this project was the participatory action research approach involved. In particular persevering with this study resulted in the inclusion of an otherwise disregarded and vulnerable population and fostered the development of authentic relationships.

Compared with other vulnerable population research, the retention rate in this study was very high. The development of rapport and a genuine relationship through ongoing connections has been identified as instrumental in creating “hope” for participants (Rodriguez & Brown, 2009; Hall, 1992, Karabanow et al., 2016) which improves the engagement of research subjects. Participants considered “regulars” at the YAC had formed relationships with the staff, were easiest to locate, and were still accessing the YAC six months after the initial interview. Participants lost to follow-up during the first and second interviews were mostly not considered YAC regulars, or did not access the YAC and had been recruited through word of mouth. Also, the issue of “aging out” is noteworthy; some participants may have been lost because they became too old to access the YAC, although, as stated earlier, other locations were arranged to maintain contact.

E-mail was found to be the second most effective strategy with the homeless youth population. The participants, themselves, identified email as an important piece of tracking information. Although some participants did not check their email often, it was one of the more reliable contact methods, in comparison with phone numbers which often change. At times, responses would be received several months after attempting to contact participants by email, which demonstrates that with time email is an effective method. Email was also useful in maintaining contact with participants who were transient as in general they would keep the same email address, but frequently changed their phone numbers.

In this study, contact with most youth participants was maintained through working with the YAC and utilizing email. High retention was also made possible by using strategies involving documenting and organizing the tracking/contact information, and by giving out items with the study logo on it. Contacting some participants was a challenge, but was often successful with time and persistence. Though not considered for this study, the profusion and popularity of social media applications accessible to youth, such as Facebook, Instagram, and WhatsApp, offer other creative methods to further improve retention of homeless youth.

Limitations

As discussed earlier, multiple strategies were used to retain participants. However, some lack of detail in contact field notes made it difficult to discern which retention strategy was responsible for locating some participants. Field notes and tracking methods should be more detailed in future projects in order to discern tracking methods for every participant at each interview. In many cases, emails would ask participants to either respond or to “drop by the Youth Action Centre”; therefore it was sometimes unclear whether it was the research team’s efforts or the YAC’s outreach that was the successful method of contact. In the future, detailed accounts of strategies used, as well as asking participants at each interview to specify which method located them, would be helpful to provide a comprehensive explanation of successful strategies.

Implications

Developing and maintaining relationships with social service agencies from the planning stages of the project is important. Agencies proved to be very helpful when aware and involved in the project. As outlined earlier, consistently using the same staff in outreach efforts helps to build relationships with the agencies and their staff, and assists in tracking. Furthermore, due to the research team’s involvement in other research projects in the city, positive collaborative relationships with agencies existed prior to this particular project, which only aided the tracking efforts.

Flexibility of research staff working hours and time devoted to outreach, as well as budgeting adequately for this is important when locating participants of this population. Patience and effort are paramount. It also proved incredibly important to use multiple strategies in locating, as in this study some youth participants preferred contact through a local agency while others preferred alternative methods of communication.

The utility of retention strategies and positive working relationships was proven by a separate one-year follow up study subsequent to the main study funded by the Homelessness Partnering Strategy (HPS). In this study 122 youth (aged 16–24 at baseline) were retained from the earlier study sample to explore innovative prevention and service programs aimed at improving housing and mental health supports for homeless and street-involved youth. The lower sample number is accounted by the fact that by this time some of the original participants had already completed their final interviews.

Conclusions

Strategies such as email, outreach, phone, and leaving messages with contact people were necessary, as they allowed for the inclusion of participants who were not accessing these agencies. As noted by Karabanow et al., (2016), maintenance of the traditional research dichotomies of participant and researcher becomes more difficult within longitudinal studies. Retaining a sample of homeless youth is difficult, however, with time, patience and effort it has proven possible. Retention success rates can be improved upon through tracking and using all possible location information and the development of relationships. Relationships with community agencies were integral to retaining the study sample, as it enabled participants to stay connected with the research team. To engage homeless youth in meaningful research as a sub-population it is acceptable, with their explicit consent at baseline, to employ strategies to locate them across time.