Predictors of Follow-Up Completion Among Runaway Substance-Abusing Adolescents and their Primary Caretakers

Abstract

Follow-up rates reported among longitudinal studies that focus on runaway adolescents and their families are relatively low. Identifying factors associated with follow-up completion might be useful for improving follow-up rates and therefore study validity. The present study explored how individual- and family-level constructs, as well as research project activities, influence the follow-up completion rate among runaway adolescents (N = 140) and their primary caregiver. Results showed that follow-up completion rates decreased as the number of research assistants (RA) assigned to each case increased and as participants’ address changes increased. Additionally, among adolescents, more frequent alcohol use was associated with lower follow-up rates. The current findings suggest that researchers should (1) design their research so that one RA is assigned to each specific case, and (2) adjust their retention strategies to account for the differences in follow-up rates based upon the participants’ drug of choice and residential stability.

Introduction

A high response rate is important when assessing behavioral change in longitudinal research (Cottler et al. 1996). Every lost subject increases the possibility of biased data or missed statistical relationships; the only way to maximize the internal and external validity of the research is to retain as many clients as possible (Hunt and White 1998). However, keeping client drop-out to an absolute minimum is difficult, particularly among marginalized populations. Although a follow-up rate of 70–80% is generally considered minimally acceptable for treatment outcome research (Festinger et al. 2008), many studies focusing on marginalized groups struggle to achieve this threshold.

Runaway Adolescents

One marginalized group that has received increased research attention in recent years is runaway adolescents and their families. In this paper, runaway adolescents are defined as adolescents residing at a runaway crisis shelter (Pollio et al. 2000). These adolescents include those who left home voluntarily as well as those who were asked to leave (and taken to the shelter) by their parents. Shelter-recruited and street-recruited youth are considered different subgroups of the runaway and homeless youth population who face significantly different life challenges (Robertson and Toro 1999). Youth recruited from runaway shelters tend to be younger than street-recruited youth and often have never spent a night on the streets (Robertson and Toro 1999). Between 72 and 87% of adolescents who seek services from a runaway shelter return home (Peled et al. 2005; Thompson et al. 2000, 2001), whereas street-recruited youth tend to avoid shelters, and returning home is not usually an option (DeRosa et al. 1999). Given the different service needs and challenges of shelter-recruited adolescents (referred to here as runaways) compared to street-recruited youth, this paper focused on runaways.

Runaways have many unique individual and family characteristics. In particular, compared to their non-runaway counterparts, runaways report high rates of substance use, depression, and delinquency (Kipke et al. 1997; Unger et al. 1997) and have been described as ‘difficult to track’ (Pollio et al. 2000). Longitudinal studies focused on runaway adolescents report follow-up rates as low as 50–57% at 3 months (Rotheram-Borus et al. 2003; Thompson et al. 2000). The low follow-up rates achieved by researchers highlight the importance of understanding barriers towards achieving adequate follow-up rates with the goal to develop better retention strategies. The current study explored how individual- and family-level variables, as well as research project activities, influence follow-up completion among runaway adolescents and their primary caregiver (PC).

Follow-Up Completion Research

The majority of research on follow-up completion has focused on strategies utilized by research teams to increase follow-up rates. In general, these studies conclude that when working with difficult-to-track populations, follow-up rates can be improved by using a variety of creative tracking strategies (Meyers et al. 2003; Robinson et al. 2007). In general Robinson et al. (2007), identified a range of strategies, from 3 to 42, used by research teams across 21 studies. Follow-up completion rates rose with the number of strategies used by the research team. In summary, while several studies identify tracking strategies to improve follow-up success, fewer studies have explored research participant characteristics associated with follow-up completion, even when aggressive tracking strategies are utilized.

Pollio et al. (2000) explored the tracking strategies used by three Midwestern runaway shelters with the aim to determine how sociodemographic variables (age, gender, ethnicity, number of runaway episodes, employment, and school attendance) influence follow-up completion among runaways at 3 months post-discharge. Findings indicated that age was negatively associated with follow-up completion; no other significant relationships were identified. Pollio et al. (2000) noted that future research should evaluate other variables potentially related to follow-up completion, including frequency of substance use. In fact, research with adult samples shows that lower follow-up completion rates are associated with using more drugs and/or alcohol (Cottler et al. 1996; Hansten et al. 2000). Also Pollio et al. (2000), suggest that future research should include longer follow-up periods since follow-up tracking becomes more difficult over time and the predictors of tracking likely change. Because research indicates that trust is essential for engaging runaway adolescents and their families (Slesnick et al. 2000), working with the same research assistant (RA) over time might aid in the development of a trusting relationship and facilitate follow-up completion. Therefore, the number of RA’s assigned to each client was evaluated in the current study as well.

Family Environment

More research is needed to determine how other variables, especially family factors, influence retention in longitudinal research. The family environment of runaways is often characterized by high levels of conflict, low levels of cohesion (Thompson et al. 2003) and residential instability (Whitbeck and Hoyt 1999). Family-level constructs such as conflict, cohesion, and changes in residence might influence the follow-up completion rate for both adolescents and their primary caretakers (PCs) as the stressors related to their family environment might take precedence over research involvement. In support of this, research among non-runaway substance abusing adolescents has shown that substance abuse in the home, inconsistent parenting, family conflict, including ongoing child abuse, and delinquent behavior of at least one family member differentiated difficult-to track and easy-to-track adolescents (Meyers et al. 2003). Furthermore, those families with less stability, such as families living below the poverty line, are more difficult to track compared to more financially secure and residentially stable families (Ensminger et al. 1997).

In addition to a lack of research on how family-level constructs influence follow-up completion among runaways, no study to date has examined factors that influence follow-up completion among the parents or PCs of runaways. This is an important inquiry since predictors of follow-up completion might differ among adolescents and PCs. If differences are observed, retention or tracking strategies might need to be tailored to each family member based upon his or her role in the family.

Current Study

In summary, previous research suggests that improving follow-up rates in longitudinal studies is essential for accurately measuring change. Prior studies conclude that numerous tracking strategies can be effective at improving follow-up rates. However, little research on follow-up completion rates has been conducted on marginalized populations such as runaway adolescents. Loss of study participants at follow-up poses a significant threat to study validity and more research examining this common threat is needed (Robinson et al. 2007). This study addressed gaps in prior research by (a) exploring the influence of alcohol and drug use on follow-up rates, (b) assessing factors associated with follow-up completion for both the adolescent and their PC, and (c) examining predictors of follow-up completion across six time points in a two-year period. Determining how these variables influence follow-up completion can augment efforts to improve follow-up completion above and beyond a research project’s tracking protocol.

Methods

Participants

All participants (140 dyads, N = 280) were recruited from the only crisis shelter for runaway adolescents in Columbus, Ohio. Data for the current study were collected as part of a larger project evaluating treatment outcomes among runaway adolescents and their families. In order to be eligible for the study, youth had to be between the ages of 12 and 17 years, residing at the runaway shelter, have the legal option of returning to a home situation, meet DSM-IV criteria for substance abuse or dependence, and have a PC willing to participate and complete the assessment instruments. Demographic characteristics are shown in Table 1.

Table 1.

Demographic characteristics of the sample (n = 140 dyads)

Variable	Youth n (%)	PC n (%)
Gender
Male	68 (48.6)	22 (15.7)
Female	72 (51.4)	118 (84.3)
Ethnicity
White, non-Hispanic	44 (31.4)	42 (30.0)
African American	88 (62.9)	85 (60.7)
Other	8 (5.7)	4 (2.9)
Not reported	–	9 (6.4)
	M (SD)	M (SD)
Age	15.5 (1.2)	41.5 (8.7)
# Follow-ups completed	4.4 (1.9)	4.4 (2.0)

Materials

A locator form was completed at baseline to assist research staff in contacting the research participant for subsequent follow-up assessment interviews. The form was updated with new contact information each time the participant completed a follow-up assessment. In particular, the participant’s current address, phone number(s) and names and phone numbers of locators or collateral contacts (friends, family, probation officers, social workers, etc.) were obtained. Adolescents were able to report up to three locators/collateral contacts during each assessment and PCs were able to report up to two locators. Separate forms were completed for the adolescent and the PC because the adolescent and the PC did not always reside at the same address at the time of the follow-up interview. In order to determine the number of address changes across the 24-month assessment period, the number of different addresses provided on the locator forms was counted (range 0–6).

The number of research assistants (RAs) completing each client’s follow-up assessment interview was determined using a project database in which RAs were required to enter both the date in which the follow-up was completed and who completed the follow-up interview. Therefore, the number of RAs was calculated by a simple count of the number of different RAs completing an assessment for each participant throughout his or her involvement in the research project.

The Form 90 Substance Use Interview (Miller 1996) was administered to the adolescent and was used to assess his or her alcohol and drug use in the prior 90 days. The Form 90 is a semi-structured interview which uses the timeline follow-back method to obtain daily use of alcohol and other illicit drugs. Test–retest correlations ranged from .62 to .99 for the different drugs of abuse among runaways (Slesnick and Tonigan 2004).

The Family Environment Scale (FES; Moos and Moos 1986) was administered to the adolescent and the PC in order to assess perceptions of the family environment and relations. The FES consists of ten subscales which measure the social-environmental characteristics of families. The conflict and cohesion subscales were used in the present analysis to assess family disturbance as these two areas have been shown to predict negative communication exchanges in delinquent and clinic-referred families (Mas 1986). In the current study, internal consistencies of conflict and cohesion ranged from .55 to .73.

Procedure

RAs approached every youth admitted to the shelter to determine interest and eligibility in the study. Permission to contact the adolescent’s PC was obtained from interested youth. PCs were contacted and, if he or she provided consent, assent was obtained from the adolescent. Of the 467 youth who were approached, 62.7% (n = 293) were eligible, and 61.4% of those eligible (n = 180 dyads) were enrolled into the program. Since data entry and cleaning are still in progress for the 180 youth and their PCs, the first 140 dyads had complete data at the time of the analysis and comprised the sample for the current study. Both PC and adolescent assessment interviews were scheduled within 24 h of recruitment, when possible. In general, adolescent interviews were conducted in private offices at the shelter while PC interviews were conducted in their home. The baseline adolescent assessment required approximately 3 h to complete and the baseline PC assessment required approximately 1½ h to complete. All participants were reimbursed for their time. Adolescents received a $40 gift card to a local retail store and parents/PCs received $25 cash after completion of the baseline assessment. All participants were compensated in cash, $40 for adolescents and $25 for parents/PCs, after completing each follow-up assessment.

As part of the larger study, families were randomly assigned to one of three project interventions [Ecologically-Based Family Therapy (EBFT), the Community Reinforcement Approach (CRA), or Motivational Enhancement Therapy (MET)]. Participants assigned to either EBFT or CRA were offered 12 therapy sessions and 2 HIV prevention sessions while participants assigned to MET were offered 2 therapy sessions and 2 HIV prevention sessions. Participants had up to 6 months to complete therapy and were tracked for a total of 2 years with interviews conducted at 3, 6, 9, 12, 18 and 24 months post-baseline. Treatment modality was controlled for in the current study in order to avoid potential confounding effects associated with determining predictors of follow-up completion.

The project utilized aggressive tracking strategies to optimize follow-up rates. Tracking typically began 4 weeks prior to the follow-up due date to ensure enough time to contact each client. Tracking procedures utilized by each RA included making phones calls, visiting locations frequented by the clients, calling and visiting locators, and using both postal mail and email correspondence. When contact was unsuccessful using these tracking procedures, RAs increased the frequency of visits to the client’s residence and work/home and checked public records for possible incarceration. This study was approved by the Institutional Review Board at The Ohio State University.

Results

Overview of Data Analysis

Multivariate linear regression analyses (95% CI) were conducted to estimate the follow-up completion rates separately for the adolescent and PC. In order to control for age, gender, and treatment modality, these variables were entered at the initial step of the regression model. The predictors of the adolescent follow-up completion rate were: number of different RAs that completed an assessment interview with the adolescent, number of locators provided at baseline, number of address changes, number of runaway episodes, FES conflict and cohesion scores, and percent days alcohol and drug use. The predictors for the PC follow-up completion rate were: number of different RAs that completed an assessment interview, number of locators, number of address changes, number of runaway episodes of the adolescent, and their FES conflict and cohesion scores.

Adolescent and Primary Caregivers’ Characteristics

Table 1 presents the characteristics of participants. Of the 140 adolescents, 48.6% (n = 68) were male, while only 15.7% of the PCs were male (n = 22). The average age was 15.5 years for adolescents and 41.5 years for PCs. The average number of follow-ups completed was 4.4 assessments for adolescents and 4.4 assessments for PCs (range = 0–6).

RA Characteristics

A total of 11 RAs were involved in engagement and follow-up tracking of the participants in the current study. Of the 11 RAs, 10 were female. Seven RAs were White, non-Hispanic and the other four were African-American. The educational level of the RAs varied and included two postdoctoral fellows, six graduate students, two post-bachelor level employees and one undergraduate student.

Relationships Among Variables

Bivariate correlations of all continuous variables were calculated separately for adolescents and PCs. For adolescents, significant correlations were found between (1) follow-up completion and number of RA changes (r = −.80; p < .001); (2) follow-up completion and number of adolescent address changes (r = −.63; p < .001); (3) follow-up completion and family conflict (r = .23; p < .05); and (4) family conflict and cohesion (r = −.51; p < .001).

For PCs, significant correlations were found between (1) follow-up completion and number of locators provided by the PC (r = −.76, p < .001); (2) follow-up completion and number of address changes for the PC (r = −.67, p < .001); (3) number of RA changes and number of address changes for the PC (r = .61; p < .001); and (4) family conflict and cohesion (r = −.67, p < .001).

Predictors of Follow-Up Completion for Adolescents

In the multivariate linear regression model for adolescents, a higher number of RAs completing an adolescent’s assessment in the 24 months (β = −0.36; p < 0.001), a higher number of address changes (β = −0.22; p < 0.001), and higher percent days of alcohol use (β = −0.52; p < 0.01) were significantly associated with lower follow-up completion rates (see Table 2). However, adolescent’s age, gender, number of locators provided, number of runaway episodes, FES cohesion, and percent days drug use did not predict follow-up completion rates. The full model explained 63% of the variance in follow-up completion (F = 12.90, p < 0.001).

Table 2.

Multiple linear regression and tolerance test for predictors of follow-up completion among adolescents and PCs

	Youth				PC
	B (SE)	β	T	TOL	B (SE)	β	T	TOL
Step 1: predictors
Constant	74.84(41.81)		1.79		87.55(21.19)		4.13***
Age	1.11(2.67)	−0.05	−0.42	.98	−0.13(0.37)	−0.05	−0.36	.89
Gender	6.95(5.95)	0.13	1.17	.99	5.70(7.67)	0.09	0.74	.99
Treatment modality	4.95(3.95)	0.14	1.25	.98	−2.48(4.23)	−0.08	−0.59	.88
Step 2: predictors
Agency-related variables
# Locators baseline	0.58(1.34)	0.03	0.43	.89	2.36(2.73)	0.08	0.86	.86
# RA Changes	−0.36(0.06)	−0.51	−5.91***	.64	−0.18(0.29)	−0.31	−2.72**	.52
Personal and family variables
# Address changes	−0.22(0.05)	−0.34	−4.15***	.70	−0.40(0.09)	−0.51	−4.51***	.54
# Times runaway	0.25(0.28)	0.06	0.89	.92	0.18(0.29)	0.06	0.64	.93
FES-conflict	1.55(1.12)	0.12	1.38	.62	−1.64(1.21)	−0.13	−1.36	.70
FES-cohesion	−0.64(1.01)	−0.06	−0.64	.58	−0.52(1.03)	−0.05	−0.51	.67
% Days drug use	0.03(0.07)	0.03	0.46	.85	–	–	–	–
% Days alcohol use	−0.52(0.18)	−0.21	−2.79**	.87	–	–	–	–
Final model
F	12.90***				9.60***
R Square	0.68				.59
Adj. R Square	0.63				.53

Note

^*p < .05;

^**p < .01;

^***p < .001

Due to the relatively large amount of variance accounted for in this model, it was important to rule out possible multicollinearity effects. Multicollinearity occurs when predictor variables are highly correlated with each other, making it difficult to determine which individual predictor variable produces the effect on the dependent variable (Bollen 1989). In order to determine if multicollinearity influenced the current study’s findings, a tolerance test was conducted. Tolerance test results range from zero to one, with results closer to one suggesting little multicollinearity. In practice, a tolerance test less than 0.1 suggests a problem with multicollinearity. Results from the tolerance test can be found in Table 2. For adolescents, tolerance test results ranged from 0.62 for conflict and cohesion to 0.99 for both age and gender, indicating little problem with multicollinearity.

Predictors of Follow-Up Completion for PC

In the multivariate model for the PC, a higher number of RAs completing the PC’s assessment in the 24 months (β = −0.18; p < 0.01) and more frequent address changes (β = −0.40; p < 0.001) were significantly associated with lower follow up completion rates (see Table 2). The other variables in the model, including PC’s age, gender, number of locators provided, number of times the adolescent ran away, and FES conflict and cohesion did not predict follow-up completion. The full model with all variables explained 53% of the variance in follow-up completion (F = 9.60, p < 0.001; see Table 2). As with the model for adolescents, a tolerance test was conducted for this model to rule out effects from multicollinearity. Tolerance test results ranged from 0.52 for RA changes and number of address changes to 0.99 for number of runaway episodes. See Table 2 for tolerance test results.

Discussion

Obtaining adequate follow-up rates when conducting longitudinal research is important for increasing study validity and, thus, increasing the confidence in conclusions that can be drawn from the findings. The present study expanded upon prior research by exploring how individual and family factors, along with previously unexplored project characteristics such as number of RA changes, influence follow-up completion among a difficult-to-track sample of runaway adolescents and their PCs. This study found that a higher number of RA changes and a higher number of address changes for both adolescents and their PCs were associated with a lower number of completed follow-up assessments. In general, predictors of follow-up completion were similar among PCs and adolescents. However, among adolescents, higher alcohol use was associated with lower follow-up rates. Several important implications can be drawn from these findings.

In regard to follow-up completion rates, as the number of RAs completing assessments with each client increased, follow-up completion decreased. Follow-up completion might be improved if research project coordinators assign the same RA to complete each subsequent assessment interview with a participant. When an RA conducts the initial engagement and assessment interview with a participant, it is possible that an alliance develops and trust builds (e.g., Martin et al. 2000) resulting in a greater likelihood that the participant will divulge information, such as address changes. This ‘single point of contact’ might therefore improve follow-up rates. Similarly, RAs might track participants more diligently when they feel connected to them. Although retaining RAs over the course of a multi-year project can be difficult, especially when working with a high risk sample, hiring those who are invested in the population of interest might increase RA retention (and thereby improve follow-up rates).

Participant characteristics also impacted follow-up completion rates. Among adolescents, as alcohol use increased, follow-up completion decreased. However, other drug use did not predict follow-up completion. This finding diverges from research with adult samples which suggests that follow-up completion is influenced by both alcohol and drug use (Cottler et al. 1996; Hansten et al. 2000). Although more research is needed to understand this finding, more severe alcohol abusing youth might isolate themselves more than drug abusing youth do and thus avoid contact with research staff.

Previous research suggests that obtaining locator information is important for ensuring adequate follow-up rates; locator information can enhance the RAs’ ability to track clients with whom they have lost contact (Meyers et al. 2003; Robinson et al. 2007). However, the current findings do not support prior research; no relationship between the number of locators provided by participants and follow-up completion rates was found. The discrepancy in findings might be attributed to the population under study. Unlike more stable populations, runaway adolescents and their families tend to move frequently. Anecdotally, locators who were contacted to assist research staff in locating participants were often surprised to hear that the family member was no longer at the prior address or phone number. Future research might confirm that maintaining a connection with each participant throughout the duration of the research project is more important than obtaining locator information for ensuring follow-up success, at least among residentially unstable families.

Limitations

Some limitations of the current study should be considered when interpreting the findings. First, this sample was recruited from the only runaway shelter in a large Midwestern city and might not represent runaway adolescents and families in other areas of the country. Economic conditions, community support, and ethnic/racial composition vary across cities, which might influence the findings. Second, RA demographics and personality characteristics were not evaluated even though such factors may influence follow-up tracking success. Further, the frequency by which each RA used the various tracking strategies was not recorded. Future research should account for both RA characteristics and frequency of tracking strategies utilized. Third, the sample size was small. A larger sample would provide more power to detect significant relationships. Finally, the relationship between PC substance use and follow-up completion was not assessed. Since the current findings suggest that adolescent alcohol use might be an important factor to consider when developing tracking procedures, future studies should examine the role of PC substance use on follow-up completion. Another area of inquiry that was not included in the current study was the influence of the clients’ assessment of the utility of the intervention. Future research should explore this potential relationship.

Conclusions

In spite of these limitations, this is the first study to (1) examine individual and family factors associated with follow-up completion for both the runaway adolescent and his or her PC, (2) examine follow-up completion over a two-year period of time, and (3) examine the impact of adolescent substance use on follow-up completion. One of the most striking findings of the current study was that our model accounted for a large amount of variance (63% for youth, 53% for PCs) in predicting follow-up completion. This suggests that our model was successful at identifying some of the most important factors associated with follow-up completion.

In summary, the current study showed that follow-up rates are associated with both the research protocol and variables relevant to the participant’s life context. Researchers should consider the multiple factors influencing follow-up completion, such as residential instability, adolescent substance use and RA assignment and address these factors at project start-up. Although more research is needed, the findings offer two practical suggestions for how researchers can take action to improve their follow-up rates with highly mobile families. First, fewer RAs should be assigned to each specific case and second, particular care should be taken to retain problem-drinking adolescents.