Implementing Family-Centered Prevention in Rural African American Communities: A Randomized Effectiveness Trial of the Strong African American Families Program

Abstract

Efforts to disseminate evidence-based prevention programs are hampered by a lack of real-world effectiveness trials undertaken with community providers. The Strong African American Families (SAAF) program is an empirically validated intervention designed to prevent problem behavior among rural African American youth. To evaluate the effectiveness of SAAF and its implementation protocols when delivered by a community provider, we conducted a randomized, wait-list-controlled trial with outcome measurements assessed longitudinally at baseline and 6 months after baseline. A total of 465 African American youth and their parents were recruited randomly from public school lists of 5^th- and 6^th-grade students in 8 rural counties in south Georgia. Youth and parents assessed targeted outcomes in their homes. The main outcome, problem behavior vulnerability, was operationalized as a latent construct comprising three indicators: tolerance for deviance, intentions to engage in risky behavior, and affiliations with risk-taking peers. SAAF was implemented with uniformly high levels of adherence (85.5%; SD = 10.8) and attendance (M = 4.1, SD = 2.9, range = 0–7). Intent-to-treat and complier average causal effect analyses revealed significant program effects on intervention-targeted parenting practices, youth self-regulatory processes, and problem behavior vulnerability. SAAF influenced problem behavior vulnerability indirectly via effects on targeted parenting and youth processes. This study supported the effectiveness of SAAF in a community setting when a systematic implementation model supports participant engagement and intervention adherence.

Well-designed prevention studies demonstrate that family-centered interventions for youth can be efficacious in preventing a range of problem behaviors, including substance use and conduct difficulties; such interventions can also enhance youth self-regulation and improve family relationships (Kumpfer & Alvarado, 2003; Lochman & Steenhoven, 2002). Although many programs have demonstrated efficacy when tested under ideal conditions, less evidence documents their transportability to real-world environments (Rohrbach, Grana, Sussman, & Valente, 2006; Spoth et al., 2013). Existing information regarding the dissemination of efficacious, family-centered programming underscores the challenges of achieving adherence to manualized intervention protocols with community providers (Elliott & Mihalic, 2004; Gottfredson et al., 2006), surmounting barriers to engagement of family members (Brody, Murry, Chen, Kogan, & Brown, 2006; Guyll, Spoth, & Redmond, 2003; Haggerty, MacKenzie, Skinner, Harachi, & Catalano, 2006), and addressing the needs of programs that serve ethnic minority youth in diverse community contexts (Kumpfer, Alvarado, Smith, & Bellamy, 2002).

In this study, we focus on the effectiveness of the Strong African American Families (SAAF) program, a primary preventive intervention for rural African American youth. Effectiveness studies examine outcomes when interventions are implemented under conditions approximating real-world circumstances, including implementation by community-based partners and teams (Elliott & Mihalic, 2004). Although prevention scientists stress the importance of effectiveness trials prior to widespread dissemination (Pentz, 2004; Spoth et al., 2013), few programs are evaluated in this manner. Furthermore, effectiveness trials on family-centered preventive interventions for African American youth living in impoverished rural environments have not been conducted. We examined the effectiveness of the SAAF program as implemented by rural community providers in south Georgia in a randomized, wait-list-controlled prevention trial involving 465 families. In the following paragraphs, we outline the context of the SAAF community effectiveness trial, including the unique geographic and cultural contexts that informed both the program and its community implementation system.

Family-Centered Prevention for Economically Distressed Rural African American Youth

Studies consistently indicate that youth from economically disadvantaged areas experience significant challenges that affect development and engagement in adolescent risk behaviors (Aber, Bennett, Conley, & Li, 1997). Poverty and economic distress are pervasive features of life for many African American families in the rural South (Snyder & McLaughlin, 2004). Communities in this region often face chronic poverty, population decline, inadequate education programs, low educational attainment, poor healthcare, substandard housing, and high levels of crime and unemployment (Snyder & McLaughlin, 2004). More than half of African American children in the rural South live in economically distressed households (Mattingly & Bean, 2010). For families with little discretionary income, rural residence can be more challenging than is life in urban areas due to a restricted range of employment opportunities, a lack of public transportation systems, a lack of recreational facilities for youth, and difficulties in obtaining physical and mental health care (Brody, Kogan, & Grange, 2012). Many African American families in the rural South live under conditions of severe, unremitting economic stress that have the potential to take a toll on youth’s development, rendering them susceptible to risk-related behaviors and other conduct problems in adolescence (Brody et al., 2012).

Historically, residence in rural communities has protected African American youth from the risk behaviors prevalent in urban areas. Recent data, however, indicate that rural African American youth’s rates of conduct problems, substance use, and risky sex equal or exceed those of their counterparts in urban and suburban areas (Brody, Chen, Kogan, Murry, & Brown, 2010; Milhausen et al., 2003; Vazsonyi, Trejos-Castillo, & Young, 2008). These behaviors pose significant threats to adolescents’ health and well-being, increasing the likelihood of negative health repercussions and social outcomes as adults (Hair et al., 2009). Mounting evidence suggests that the consequences of engagement in risky behavior are greater for African Americans than for their European American peers. For example, African American youth experience more negative consequences per ounce of alcohol or drugs consumed than do European Americans (Gillmore, Catalano, Morrison, & Wells, 1990; Jones-Webb, 1998), and conduct problems are more likely to lead to legal problems and incarceration (Bridges & Steen, 1998). These data underscore the need for efficacious primary prevention strategies for rural African American youth as well as implementation systems uniquely designed to allow members of rural communities to access these programs.

SAAF was developed to address the lack of ecologically appropriate preventive interventions for rural African American youth. The program was informed by longitudinal studies documenting powerful factors in the family environment that protected youth’s development from disruption due to the effects of economic distress (Brody et al., 2012; Brody, Murry, et al., 2004). Caregivers of well-adjusted youth practiced involved/vigilant parenting, which included warm, nurturant behavior along with high levels of monitoring and control, adaptive racial socialization, and the establishment of clear expectations about substance use and peer involvement. Adaptive racial socialization includes messages to youth designed to instill racial pride and teach strategies for coping with discrimination. Youth whose caregivers practiced involved/vigilant parenting developed high levels of self-regulation and emotion regulation, maintained engagement with school, and developed a positive ethnic identity despite family economic challenges and exposure to racial discrimination in the community.

On the basis of these studies, intrapersonal and family protective processes associated with reduced adolescent problem behavior were translated into SAAF, a family skills training program for preadolescent youth and their caregivers. SAAF consists of seven weekly, 2-hour meetings in which parents and youth participate in separate, concurrent skill-building sessions (1 hour), followed by a family session (1 hour) in which parents and youth jointly practice the skills they learned in their separate sessions. Curriculum content for the parent and youth sessions is presented on videotapes, depicting family interactions that illustrate key concepts. Parents are taught to use involved/vigilant caregiving practices. Youth learn adaptive behaviors to use when encountering racism, ways to form goals for the future and make plans to attain them, similarities and differences between themselves and agemates who use alcohol, and peer pressure resistance strategies. Together, family members practice communication skills and engage in activities to increase family cohesion and youth’s positive involvement in their families.

SAAF is based on a developmental model of processes (see Figure 1) through which program participation has been shown to protect rural African American youth from the initiation and escalation of risk behavior in adolescence (Brody, Kogan, Chen, & Murry, 2008; Brody et al., 2004; Brody, Murry, Kogan, et al., 2006). SAAF affects risk behavior vulnerabilities by enhancing targeted parenting and youth self-regulatory processes; youth self-regulatory processes are also enhanced indirectly via intervention effects on parenting. Youth self-regulatory processes constitute a proximal mediator of changes in vulnerability to engagement in risk behavior, both contemporaneously and subsequently in middle and later adolescence.

Figure 1.

Conceptual model of SAAF program theory.

A cluster randomized prevention trial with 677 youth and their caregivers provided evidence supporting SAAF’s efficacy in deterring adolescent alcohol use and conduct problems. Implementation protocols developed for the trial yielded high levels of program adherence (90% of program activities implemented) by lay community program facilitators and high levels of participation (65% of families attended 5 or more sessions) among a typically hard to engage, economically distressed population (Brody, Murry, Chen, et al., 2006). Posttest analyses revealed significant effects on caregivers’ use of involved/vigilant parenting, d = .47 (Brody et al., 2004), and on youth intrapersonal protective processes targeted in the intervention, d = .39 (Brody, Murry, Kogan, et al., 2006). At the 2-year follow-up, SAAF participants evinced 17.4% less growth in alcohol use for each unit increase among the control group. Regarding conduct problems, control group participants were twice as likely to report problems as were SAAF participants, d =.37 (Brody et al., 2008). Intervention effects on alcohol use persisted over a period of 5 years (Brody et al., 2010). Assignment to the SAAF condition was associated with a significantly slower rate (β = .23, p < .05) of growth in alcohol use. SAAF’s potential public health impact is supported further by findings indicating that the program is efficacious with families who confront a variety of challenges, including severe economic hardship and caregiver depression (Beach et al., 2008; Brody, Murry, Chen, et al., 2006).

Transporting Family-Centered Prevention to Rural African American Families Through the Cooperative Extension Service

The capacity for SAAF to achieve widespread public health impact depends upon its potential for implementation in real-world settings. Inclusive reviews of family-centered preventive interventions reveal that, although many programs have demonstrated efficacy when tested under academic conditions, less support for effectiveness is available for evidence-based programs that community providers implement (Rohrbach et al., 2006; Spoth, Clair, Greenberg, Redmond, & Shin, 2007). Existing knowledge regarding the dissemination of efficacious, family-centered programming indicates that implementation is limited by problems with program adherence (Gottfredson et al., 2006), low participation rates (Spoth, Redmond, et al., 2007), and a lack of attention to the difficulties of disseminating programs to ethnic minority populations in resource-poor rural environments (Kumpfer et al., 2002).

To advance SAAF’s potential for public health impact while addressing existing limitations in the literature, we partnered with the Cooperative Extension Service (CES) to evaluate the effects of SAAF when delivered by local providers. This approach has proven effective in previous research (Spoth, Greenberg, Bierman, & Redmond, 2004). Prior to implementation, a Technical Assistance (TA) team met with organizational leaders in the CES to develop collaborative goals and a shared vision for SAAF implementation. A pilot test of the implementation model was conducted in a two-county region. This enabled Extension administrators and the TA team to fine-tune protocols and to develop efficient monitoring and paperwork systems. The protocols that were developed included (a) community outreach and engagement, (b) surmounting of barriers to participation, and (c) implementation of the program with high levels of fidelity. The TA team worked with area extension agents, who hired and supervised SAAF Extension Educators (SEEs)—full-time, bachelor’s level employees with experience in working with African American families in educational or prevention settings. The SEEs received training and TA from the SAAF development team at the University of Georgia, and local CES agents provided SEEs with monitoring and supervision. SEEs received (a) 16 hours of general implementation training, (b) 24 hours of training on implementing the program, and (c) monthly TA that included 2 hours of consultation via Skype and 2 to 4 hours of e-mail or phone support.

The first phase of implementation involved SEEs’ conducting outreach to schools, faith-based organizations, and other local stakeholders to develop close ties with African American communities. These ties were deemed critical to engaging participants as well as to the identification of community members who would provide the intervention. SEEs then hired trusted community members to facilitate the SAAF intervention groups and arranged training for them. SAAF program facilitators received 24 hours of training from the TA team on the SAAF curriculum and program implementation protocols. The SEEs then organized the implementation of the program by teams of three facilitators at convenient community sites. The SEEs organized transportation and child care assistance for parents who required it and ensured that each session began with a meal for families. Parents also received $20 per session to reimburse them for miscellaneous costs (e.g., missed work, non-SAAF-provided child care, and so forth). Prior to the first intervention session, the SEEs arranged a home visit with each family from a SAAF program facilitator. The facilitator showed family members a promotional video and answered questions regarding participation in the program.

During the 7-week implementation of SAAF, the SEEs supported program facilitators’ adherence to the manualized intervention protocols. Program facilitators completed debriefing questionnaires at the end of each session, which were sent to the SEEs for review. SEEs held weekly meetings with program facilitators, in person and by phone, regarding problems and successes in each group and routinely affirmed the importance of following the manualized protocols. SEEs also made random visits to sessions to support implementation and program adherence and observed three complete program sessions. During implementation, SEEs also reviewed attendance records and contacted families who experienced barriers to attending to formulate solutions and encourage continued participation.

Summary

To evaluate the effectiveness of SAAF, we conducted a randomized wait-list-control trial. CES providers in eight rural counties in south Georgia implemented SAAF over a 3-year period, with training and technical assistance from the SAAF development team. We hypothesized that, within the CES system, SAAF would (a) be delivered with high levels of program adherence and (b) deter preadolescent youth’s risk behavior vulnerability by enhancing intervention-targeted youth and family protective processes.

Method

Participants

Youth and their primary caregivers (N = 465) were recruited from eight rural counties in Georgia, each of which was served by a local Cooperative Extension office. School districts provided lists of African American 5th- and 6^th-grade students, whose parents were contacted in random order to discuss participation. Eligibility requirements included the presence in the family of a youth 11 or 12 years of age at pretest who self-identified as African American. Self-reported African American race was not a requirement for primary caregivers; however, all participating caregivers self-identified as African American. Of the 594 families screened for eligibility (see Figure 2), 563 were eligible to participate; of these, 465 were enrolled in the project (83%) and were randomized to condition. Approximately 52% (242) were assigned to receive SAAF beginning within 4 weeks; 84% of them received a pre-intervention program information visit. The remaining 223 families were wait-listed for 1 year. Retention from pretest to posttest was 88.4% for the SAAF group and 90.6% for the wait-listed group. Families who dropped out of the study did not differ from those assessed at posttest on any study variables.

Figure 2.

Flow of participants through the trial.

Procedures

Research staff initially contacted families with a letter introducing the study. Follow-up phone calls and in-person contacts were made to families by the local SEE. At baseline and at a 6-month posttest (administered approximately 3 months after the intervention program concluded), African American field researchers visited the families’ homes to collect data from youth and primary caregivers using audio computer-assisted self-interviews on laptop computers. Informed consent/assent was obtained at pretest. To compensate them for their time, caregivers were paid $50.00 and youth were paid $25.00 at each assessment. All study protocols were approved by the University of Georgia’s Institutional Review Board.

Implementation of SAAF Within the CES

Teams of three African American community members conducted 26 SAAF groups for those families assigned to receive SAAF in the near term. All sessions were videotaped. For each group, two caregiver, two youth, and two family sessions were selected randomly and scored for adherence to the prevention curriculum. Reliability checks were conducted on 20% of the adherence assessments; the intraclass correlation between judges was .78. The mean coverage of session components was 84.5% (SD = 10.8). Approximately 57% (137) of the families in the SAAF group attended 5 or more intervention sessions, 10% (23) attended 3 or 4 sessions, 9% (21) attended 1 or 2 sessions, and 25% (61) did not attend any SAAF sessions. The mean session attendance was 4.1 (SD = 2.9).

Measures

Intervention-targeted parenting

Parenting was assessed at baseline and posttest with measures used previously in our research. Youth and parents completed the nine-item involved/vigilant parenting scale (Brody, Murry, et al., 2004), which assesses caregivers’ use of monitoring, consistent discipline, and nurturant parenting practices (e.g., “How often does your [parent] know who you are with when you are away from home?”). Cronbach’s alpha exceeded .62 for youth and .63 for parents. Parents and youth completed parallel versions of a nine-item scale assessing parental discussion with the youth of expectations regarding risky behavior (e.g., “I have explained my rules concerning alcohol use to my child”). Cronbach’s alpha exceeded .94 for parents and .95 for youth. Parents also reported on relationship harmony and distress (e.g., “Your child is easy to get along with”) using the Interaction Behavior Questionnaire (Prinz, Foster, Kent, & O'Leary, 1979); alpha exceeded .90. The scores on these scales were standardized and summed to provide a multireporter index of targeted parenting behavior. Reliability for our composite measure was .91.

Intervention-targeted self-regulatory processes

Youth completed a four-item anger control scale (e.g., “When I am feeling mad, I control my temper”). Cronbach’s alpha was .84. A seven-item scale (e.g., “You can easily calm down when you are excited or wound up”) indexed behavioral self-control (Wills, McNamara, Vaccaro, & Hirky, 1996). Cronbach’s alpha exceeded .70. Youth also completed the Hays and Ellickson (1990) resistance efficacy measure, which comprises three scenarios in which a youth is offered alcohol, cigarettes, or marijuana. Youth reported their likely reactions (1 = I would take the cigarette and smoke it; 2 = I would say, "Not now, maybe some other time"; 3 = I would tell my friend "No" and not smoke it). Alpha for the scale was .74 at pretest and .72 at posttest. The scales were standardized and summed to provide a multimeasure index of intervention-targeted self-regulatory processes; Cronbach’s alpha for the multimeasure composite was .82.

Risk behavior vulnerability

In the present study, we operationalized risk behavior vulnerability as a latent construct using three scales with robust links to risky behavior in middle and late adolescence. Youth completed the Tolerance for Deviance scale (Jessor, Turbin, & Costa, 1998); items included, “How often is it okay for someone your age to…ruin or damage something on purpose/cheat on school tests/smoke marijuana?” Alpha was .87 at pretest and .86 at posttest. A 13-item inventory assessed intentions to use substances and engage in sexual activity (e.g., “Sometime in the next 12 months, I will probably smoke marijuana”). Alpha was .88 at pretest and .86 at posttest. Youth reported on their affiliations with risk-taking peers on a 12-item measure (e.g., “How many of your close friends have smoked cigarettes/gotten into fights with other kids/had sex?”) used in our previous research (Brody et al., 2001). Alpha was .80 at pretest and .81 at posttest.

Control variables

Economic hardship was measured using four items to assess the adequacy of a family’s income to meet material needs such as food, shelter, medical care, and clothing. Cronbach’s alpha was .85. Parents completed a demographic interview that included household composition, household income, and their own educational attainment. An SES index was defined as the sum of standardized values for household income and parental educational attainment. Poverty status was determined based on federal guidelines. Intervention status, single-parent family structure, caregiver’s employment status (unemployed versus employed) and youth gender were dummy coded.

Data Analysis

Preliminary analyses included evaluation of attrition and equivalence between experimental groups, and a confirmatory factor analysis (CFA) of the latent vulnerability construct. Consistency of the implementation across eight communities was evaluated by examining mean differences by county and year of implementation for participant attendance and intervention adherence. Intent-to-treat analyses of SAAF effectiveness were conducted with structural equation modeling (SEM) as implemented in Mplus. We estimated intervention effects at posttest, controlling for baseline levels of risk behavior vulnerability, intervention-targeted parenting, and youth self-regulatory processes. We then tested the SAAF program model, as presented in Figure 1, using boot-strapping to estimate indirect effects. Because intent-to-treat analyses underestimate intervention effects (Gupta, 2011), we repeated the analysis using complier average causal effect modeling (CACE; Chen, Geng, & Zhou, 2009). CACE uses a mixture model to identify latent groups of compliers and noncompliers within the control and treatment groups. Causal effects are then determined by comparing compliers in both treatment and control groups. Finally, we determined whether program effects were similar across county of implementation and within each intervention group using the multilevel analysis in Mplus to examine variation by county or intervention group in effectiveness.

Results

Preliminary Analyses

Table 1 presents descriptive statistics for the entire sample, as well as comparisons of demographic and study variables between the SAAF group and the wait-list control group. Although most (60.2%) of the primary caregivers were employed, 62.7% of the participants lived below federal poverty standards. These families were representative of the area in which they lived (Boatright, 2009). At baseline, the families randomized to SAAF differed from the wait-list control families on one characteristic: SAAF families were experiencing slightly more economic hardship. This variable was controlled in all analyses. Comparisons of participants who did not complete the posttest vs. those who did (see Figure 2) revealed no associations with any study variables, including treatment condition.

Table 1.

Sample Characteristics, Pretest Equivalence, and Posttest Differences by Experimental Condition

Measures	Total a	Control b	Treatment c	t-value	P	Cohen’s d
Demographic characteristics (pretest)
Economic hardship	10.41 (2.89)	10.12 (2.88)	10.67 (2.88)	2.05	0.04	--
SES	−0.13 (8.43)	−0.05 (0.82)	0.02 (0.86)	0.98	0.33	--
Single-parent family structure	0.62 (0.49)	0.62 (0.49)	0.62 (0.49)	−0.17	0.87	--
Unemployment	0.61 (0.49)	0.64 (0.48)	0.57 (0.50)	−0.27	0.79
Living below poverty level	0.77 (0.42)	0.77 (0.42)	0.76 (0.43)	−1.48	0.14
Measures of risk behavior vulnerability
Tolerance of deviance
Pretest	18.67 (5.01)	19.11 (6.30)	18.26 (3.39)	−1.80	0.07	--
Posttest	18.92 (5.60)	19.83 (7.10)	18.05 (3.48)	−3.21	0.01	0.32
Risk behavior intentions
Pretest	14.43 (3.63)	14.59 (4.01)	14.30 (3.25)	−0.86	0.39	--
Posttest	14.80 (4.15)	15.19 (4.83)	14.42 (3.34)	−1.89	0.06	0.19
Peer behavior
Pretest	0.04 (6.66)	0.46 (6.96)	−0.35 (6.36)	−1.31	0.19	--
Posttest	−0.19 (6.46)	0.39 (7.12)	−.73 (5.71)	−1.77	0.08	0.17
Mediators
Self-regulation index
Pretest	0.00 (2.17)	−0.02 (2.21)	0.02 (2.14)	0.21	0.84	--
Posttest	0.00 (2.17)	−0.30 (2.34)	0.28 (1.95)	2.75	0.01	0.27
Parenting index
Pretest	0.00 (3.07)	0.05 (3.04)	−0.04 (3.11)	−0.31	0.75	--
Posttest	0.01 (2.96)	−0.32(3.24)	0.28 (2.63)	2.08	0.04	0.20

^aN = 465.

^bn = 223.

^cn = 242.

We next examined the adequacy of the measurement model for the latent vulnerability construct at baseline and posttest with CFA. The model fit the data well (χ² = 18.99, p = .01; CFI = .99; SRMR = .02), with all indicators loading significantly (λ >.64, p < .001) in the expected directions.

Variability in implementation across project sites poses problems for interpreting trial data (Brody et al., 2004). Implementation had the potential to vary based on three characteristics. First, each SEE administered the intervention in two counties. Thus, these two-county units may have varied on intervention fidelity and participant attendance as a function of individual SEEs’ performance. Second, the individual counties may have had unique properties that affected intervention implementation. Finally, the year in which the intervention was received (Year 1, Year 2, or Year 3) may have affected implementation as teams became more experienced and acquired greater expertise in presenting the intervention. General linear models examining mean differences in fidelity and participant attendance revealed no differences across these characteristics. This suggests that the implementation model was successful in facilitating a uniform experience for participants across the provider network and across the duration of the project.

ITT analyses

We investigated SAAF effectiveness on risk behavior vulnerability, intervention-targeted parenting, and intervention-targeted youth self-regulation in separate analyses with baseline levels controlled. Intent-to-treat analyses revealed significant effects on risk behavior vulnerability (β = −.10, p < .01; d = .30), intervention-targeted parenting practices (β = .11, p < .01; d = .40), and youth self-regulatory processes (β = .11, p < .01; d = .50). Figure 3 presents the SEM of SAAF’s theoretical model, exploring the indirect effects of SAAF on risk behavior vulnerability. The model fit the data as follows: χ² = 177.25, df = 69, p = .00; CFI = .94; and RMSEA = .05. Table 2 summarizes the indirect effects. As hypothesized, program effects were mediated through self-regulatory processes, which were affected directly by participation in SAAF and indirectly via SAAF effects on targeted parenting.

Figure 3.

ITT and CACE analyses of SAAF program theory; χ² = 177.25, df = 69, p = .00, CFI=.94, and RMSEA = .05. Values presented are standardized parameter estimates. Economic hardship, male gender, SES, and single-parent family structure are controlled. N = 465. Parameters in parentheses are for the CACE model.

*p ≤ .05. **p ≤ .01.

Table 2.

Indirect Effects of SAAF on Risk Behavior Vulnerability

Predictors	Mediator	Outcome	Indirect Effect	95% CI	PM
Group	Self-regulation	Risk behavior vulnerability	−.284	[−.653, −.071]	.28
			−1.092	[−2.385, −.355]	.20
Group	Parenting	Risk behavior vulnerability	−.107	[−.302, −.016]	.11
			−.319	[−.972, −.009]	.06
Parenting	Self-regulation	Risk behavior vulnerability	−.090	[−.165, −.042]	.34
			−.067	[−.188, −.012]	.23
Group	Parenting	Self-regulation	.089	[.028, .176]	.16
			.170	[.028, .452]	.08
Group	Parenting, Self-regulation	Risk behavior vulnerability	−.055	[−.130, −.017]	.05
			−.095	[−.341, −.015]	.02

Note. Estimates for compliers in CACE model are presented in italics. P_M = the ratio of the indirect effect to the total effect.

CACE analysis

We replicated the ITT analyses using a CACE model with dose (in the complier group), economic status, youth gender, and family structure as covariates for estimating latent compliance groups. Typical fit statistics are not available for mixture models. The entropy value, which indexes the models’ classification accuracy, exceeded .75, an acceptable value according to Nagin (1999). The model created a group of compliers in the SAAF condition and an equivalent group of compliers in the wait-list-control condition. The influence of experimental assignment in the latent compliance group revealed significant main effects on risk behavior vulnerability (β = −.35, p < .01; d = .83), intervention-targeted parenting practices (β = .20, p < .01; d = .48), and youth self-regulatory processes (β = .35, p < .01; d = .83). In Figure 3, the CACE estimates of effects in SAAF’s theoretical model are presented below the ITT parameters; indirect effects in the CACE model replicated the ITT findings (see Table 2).

Multilevel analyses

In a final step, we explored SAAF’s effectiveness across counties using an unconditional multilevel model. Results indicated that the intraclass correlation coefficients were small and not significant (ICC ≤ .2, ns), suggesting that no variation occurred across the eight counties in the effectiveness of the intervention on intermediate and risk behavior outcomes. We then examined the data to see if SAAF’s effectiveness was similar across the 26 intervention groups. Again, the intraclass correlation coefficients were small and not significant (ICC ≤ .006, ns), suggesting that no variation occurred across intervention groups in the effectiveness of the intervention on intermediate or risk behavior outcomes.

Discussion

Although many family-centered prevention programs have demonstrated efficacy when tested under academic conditions, less evidence documents their effective transfer to real-world environments (Rohrbach et al., 2006; Spoth, Kavanagh, & Dishion, 2002). In this study, we investigated the effectiveness of the SAAF program as implemented by rural community providers in south Georgia. A manualized implementation system was developed to present SAAF through the CES in eight rural Georgia communities. The system, which included training and routine TA, was effective in maintaining high levels of program adherence across the eight implementation sites. Effectiveness analyses revealed that SAAF significantly deterred risk behavior vulnerability in rural African American preadolescent youth; hypothesized effects on intervention-targeted parenting practices and youth self-regulatory processes mediated this effect.

Past research on SAAF within the context of a cluster randomized trial indicated, at posttest, medium effect sizes on youth self-regulatory characteristics and targeted parenting behavior (Brody, Murry, Gerrard, et al., 2006; Brody et al., 2004; Brody, Murry, Kogan, et al., 2006). In contrast, our ITT findings indicated small effect sizes on these proximal outcomes; when examined with CACE methods, medium effect sizes were detected. A diminution of effect in the context of effectiveness trials is relatively common (Curtis, Ronan, & Borduin, 2004; Hallfors, Cho, Sanchez, Khatapoush, Kim, & Bauer, 2006; Neil & Christensen, 2009). Our CACE effects, however, suggest that, for families receiving an effective dose of prevention, effects were similar to those found in the original trial. Discrepancies may indicate systematic differences in the samples in terms of characteristics of families who did not receive a full dose across the trials.

Obtaining high levels of program adherence in community provider contexts is essential to realizing program effects and public health impact (Rohrbach et al., 2006). Using an integrated system of training, TA, and manualized protocols, African American community members who were recruited and monitored by a local CES-based staff member presented SAAF with high adherence (85% of program components implemented). This level of adherence is comparable to that achieved in the highly controlled context of the SAAF trial (90%; Brody et al., 2004). In many current dissemination programs, systems for supporting adherence may be minimal after providers are trained in presenting an intervention, resulting in provider “drift” and compromised effectiveness (Rohrbach et al., 2006). In contrast, in addition to manualizing the intervention content, the SAAF developers worked with CES to create a manualized set of protocols that specified practices for ensuring ongoing program adherence and participant engagement. Program adherence was enhanced by SEEs’ careful selection, training, and monitoring of program facilitators. SEEs’ activities were supported through training, routine TA, and a paperwork tracking system designed to ensure protocol compliance. This approach is consistent with recent studies suggesting that adoption of evidence-based programs in a provider network requires more than training in a set of procedures (Spoth, Clair, et al., 2007). Rather, providers must adopt a systematic and structured set of practices that may not have been routine prior to adoption of a specific evidence-based program. The success of the SAAF effectiveness trial supports the need for an ongoing partnership between providers and program developers to formulate, refine, and troubleshoot a systematic adaptation of the existing program to achieve effective intervention (Spoth, Greenberg, Bierman, & Redmond, 2004).

Prior to the creation of a partnership with SAAF developers, few African American families availed themselves of the family enrichment services that the CES network provided in these eight rural communities. Poverty, mistrust of community providers, and a lack of familiarity with the benefits of family-centered interventions have been identified as key barriers to minority families’ participation in evidence-based preventive interventions (Brody et al., 2012). In our randomly recruited sample, 63% of the families lived at or below the federally defined poverty level. The SAAF implementation protocol is unique in specifying the need for active outreach to African American community stakeholders to facilitate family engagement. Engagement with the overall research project (only 15% of eligible families declined participation) and attendance rates were similar to those reported in the original trial (Brody et al., 2004) and exceeded those achieved in past CES programming within these communities. This level of engagement was remarkable, given the high poverty rate in the sample.

In evaluating the significance of the high participation levels attained in this effectiveness trial, it is important to consider the potential impact of the modest incentive provided to participants to defray attendance costs. In real-world prevention practice, such incentives are rare. Thus, this aspect of our protocols may have affected participation in a way that would not necessarily transfer into the context of program dissemination. A number of studies, however, suggest that provision of incentives may be necessary to ensure effective prevention implementation (Guyll et al., 2003; Heinrichs, 2006). For example, an experimental evaluation of modest incentives for parent training indicated their importance for many families to overcome economic and social barriers to participation (Heinrichs, 2006). Importantly, incentives were found to be particularly salient for families who were least likely to participate in a preventive intervention even though they were the most in need of it. From a policy perspective, these studies, as well as our findings that indicated high levels of engagement with a modest incentive, suggest that providing incentives for participation beyond the targeted benefits of the program should be considered part of a successful protocol to implement effective prevention in the community.

Our findings suggest other policy and prevention implications. The present study supported the effectiveness of SAAF in a community setting when a systematic, well-resourced implementation model supports participant engagement and intervention adherence. Examination of variability by county and by SEE indicated that the system facilitated consistent intervention implementation that proved to be resistant to provider or facilitator “drift.” Under these circumstances, engagement of significant numbers of rural African American families and exposure to an effective dose of the intervention are likely. From a prevention and policy perspective, these findings confirm the importance of well-designed and well-resourced systems for ensuring adherence, attendance, and consistent implementation. For many real-world implementers of prevention services, attaining intervention fidelity and securing high levels of attendance remain challenging. In some cases, the resources necessary to ensure effectiveness are not allocated, and provider systems focus on the quantity of programming delivered rather than the quality. Unfortunately, the effects of these efforts may be quite limited, representing in essence a net loss of resources invested. Additional research that examines cost effectiveness is indicated to support not only the importance of providing prevention services in general, but also the cost benefit of allocating sufficient funds for engagement and fidelity.

Several limitations to the study should be noted. The wait-list design precluded long-term follow-up and examination of behavioral outcomes. Nevertheless, this design, common in tests of dissemination effectiveness, was appropriate given prior evidence of SAAF’s effects on intermediary processes and strong links between intervention-targeted processes and behavioral outcomes 5 years post-intervention (Brody et al., 2010; Brody et al., 2008; Brody, Murry, Gerrard, et al., 2006). Concerns are further mitigated by the use of a robust multimeasure assessment of behavioral vulnerability with measures that have demonstrated links to substance use and externalizing problems in adolescence. Second, a modest monetary payment was made to defray attendance costs. Although an indicated practice for effectiveness trials that focus on achieving adherence, the potential for high levels of attendance without such payments is not known. Recent studies, however, suggest that incentives may be needed as a routine part of prevention practice. Finally, our measure of fidelity was limited to adherence to the manual and did not include information on facilitator competence.